-- Logs begin at Thu 2022-06-23 21:36:18 UTC, end at Thu 2022-06-23 23:07:18 UTC. -- Jun 23 22:25:54.594250 dv-cinder-813229-11-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 23 22:25:54.603105 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 23 22:25:54.605143 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 23 22:25:54 2022] *** Jun 23 22:25:54.605143 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 23 22:25:54.605143 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 23 22:25:54.605143 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: nodename: dv-cinder-813229-11-iscsi Jun 23 22:25:54.605143 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: machine: x86_64 Jun 23 22:25:54.605143 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: clock source: unix Jun 23 22:25:54.605143 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: pcre jit disabled Jun 23 22:25:54.605143 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: detected number of CPU cores: 4 Jun 23 22:25:54.605143 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: current working directory: / Jun 23 22:25:54.605143 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: detected binary path: /usr/bin/uwsgi-core Jun 23 22:25:54.605730 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: your processes number limit is 47680 Jun 23 22:25:54.605730 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: your memory page size is 4096 bytes Jun 23 22:25:54.605730 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: detected max file descriptor number: 2048 Jun 23 22:25:54.605730 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: lock engine: pthread robust mutexes Jun 23 22:25:54.605730 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: thunder lock: enabled Jun 23 22:25:54.605730 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 23 22:25:54.605730 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 23 22:25:54.649798 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: Python main interpreter initialized at 0x55978f0979c0 Jun 23 22:25:54.649798 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: python threads support enabled Jun 23 22:25:54.649798 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: your server socket listen backlog is limited to 100 connections Jun 23 22:25:54.649798 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: your mercy for graceful operations on workers is 90 seconds Jun 23 22:25:54.650141 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: mapped 403077 bytes (393 KB) for 2 cores Jun 23 22:25:54.650216 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: *** Operational MODE: preforking *** Jun 23 22:25:54.650483 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: *** uWSGI is running in multiple interpreter mode *** Jun 23 22:25:54.650483 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: spawned uWSGI master process (pid: 80511) Jun 23 22:25:54.650684 dv-cinder-813229-11-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 23 22:25:54.652056 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: spawned uWSGI worker 1 (pid: 80518, cores: 1) Jun 23 22:25:54.652056 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: spawned uWSGI worker 2 (pid: 80519, cores: 1) Jun 23 22:25:54.652056 dv-cinder-813229-11-iscsi devstack@keystone.service[80511]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 23 22:26:00.136526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 22:26:00.136526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: from cryptography.utils import int_from_bytes Jun 23 22:26:00.137336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 22:26:00.137336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: from cryptography.utils import int_from_bytes Jun 23 22:26:00.137462 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 22:26:00.137462 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: from cryptography.utils import int_from_bytes Jun 23 22:26:00.137462 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 22:26:00.137462 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: from cryptography.utils import int_from_bytes Jun 23 22:26:00.186830 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 22:26:00.186993 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 22:26:00.187086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 22:26:00.187174 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 22:26:00.187253 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] command line args: [] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 22:26:00.187329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] command line args: [] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 22:26:00.187453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 22:26:00.187545 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 22:26:00.187622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ================================================================================ {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 22:26:00.187699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ================================================================================ {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 22:26:00.187872 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] admin_token = **** {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.187952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] admin_token = **** {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.188122 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] config_dir = [] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.188212 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] config_dir = [] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.188315 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.188394 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.188496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] config_source = [] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.188593 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] config_source = [] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.188700 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] debug = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.188795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] debug = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.188897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.189212 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.189349 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.189455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.189558 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.189654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.189750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.189851 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.189953 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.190058 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.190161 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] list_limit = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.190258 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] list_limit = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.190258 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] log_config_append = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.190461 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] log_config_append = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.190563 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.190671 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.190774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] log_dir = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.190877 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] log_dir = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.190978 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] log_file = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.191086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] log_file = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.191171 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.191299 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.191380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.191496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.191570 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.191683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.191757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.191885 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.191961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.192082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.192164 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.192280 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.192366 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.192439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.192554 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.192668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.192742 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.192862 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.192946 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.193023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.193137 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.193211 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.193354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.193440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.193600 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.193695 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.193844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.193925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.194116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.194235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.194600 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.194684 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.194819 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] publish_errors = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.194897 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] publish_errors = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.195014 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.195094 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.195212 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.195301 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.195377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.195492 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.195622 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.195721 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.195856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.195933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.196088 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] standard_threads = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.196168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] standard_threads = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.196319 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.196401 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.196559 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.196669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.196763 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.196833 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.196911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] use_journal = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.196988 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] use_journal = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.197087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] use_json = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.197172 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] use_json = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.197243 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] use_stderr = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.197311 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] use_stderr = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.197420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] use_syslog = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.197503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] use_syslog = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.197646 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.197719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:26:00.197888 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.197971 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.198124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.198207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.198381 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.198457 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.198617 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.198692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.198862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.198942 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.199098 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.199175 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.199342 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.199418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.199577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.199659 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.199822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.199899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.200064 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.200139 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.200356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.200438 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.200608 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.200699 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.200858 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.201022 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.201100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.201264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.201350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.201514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.201591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.201764 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.201841 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.202003 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.202087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.202254 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.202330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.202499 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.202576 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.202734 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.202812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.202969 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.203046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.203223 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.203306 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.203457 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.203536 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.203702 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.203785 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.203932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.204023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.204203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.204296 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.204369 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.204444 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.204633 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.204712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.204875 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.204954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.205118 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.205200 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.205347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.205424 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.205575 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.205653 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.205805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.205881 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.206051 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.206159 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.206287 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.206399 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.206522 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.206625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.206750 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.206861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.206971 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.207057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.207206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.207291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.207436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.207523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.207649 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.207729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.207882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.207959 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.208111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.208229 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.208343 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.208423 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.208619 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.208744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.208830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.208915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.209055 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.209135 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.209283 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.209365 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.209514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.209596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.209745 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.209832 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.209980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.210057 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.210206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.210283 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.210440 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.210526 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.210674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.210749 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.210907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.210984 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.211140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.211225 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.211375 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.211463 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.211600 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.211682 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.211832 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.211917 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.212067 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.212146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.212301 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.212380 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.212530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.212627 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.212788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.212874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.213035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.213113 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.213264 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.213349 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.213494 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.213577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.213726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.213803 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.213962 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.214047 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.214212 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.214289 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.214455 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.214532 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.214696 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.214844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.214920 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.214996 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.215153 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.215237 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.215385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.215470 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.215623 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.215701 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.215911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.216014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.216142 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.216247 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.216389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.216471 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.216626 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.216710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.216865 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.216943 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.217099 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.217179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.217331 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.217415 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.217565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.217644 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.217824 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.217902 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.218060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.218143 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.218303 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.218381 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.218539 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.218619 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.218775 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.218853 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.219008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.219085 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.219242 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.219322 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.219477 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.219555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.219705 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.219782 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.219934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.220013 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.220162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.220240 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.220391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.220468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.220645 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.220725 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.220878 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.220956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.221109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.221185 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.221345 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.221423 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.221577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.221654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.221820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.221897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.222074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.222152 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.222312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.222396 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.222541 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] auth.external = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.222619 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.222770 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.222849 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] auth.external = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.223000 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.223077 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.223278 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.223366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.223507 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] auth.password = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.223592 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.223734 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] auth.token = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.223814 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] auth.password = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.223963 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] auth.token = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.224047 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.224195 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.224273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.224431 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.224509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.224680 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.224761 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.224923 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.225001 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.225164 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.225244 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.225393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.225472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.225626 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] credential.caching = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.225706 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.225861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] credential.caching = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.225941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.226103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.226181 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.226353 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.226480 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.226588 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.226671 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.226829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.226907 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.227062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.227141 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.227297 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.227376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.227561 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.227645 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.227793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.227879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.228043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.228123 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.228296 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.228375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.228557 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.228729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.228837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.228916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.229076 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.229160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.229330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.229418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.229565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] federation.caching = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.229656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.229798 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.229878 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] federation.caching = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.230034 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.230120 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.230275 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.230353 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.230503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.230582 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.230739 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.230825 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.230973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.231051 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.231217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.231295 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.231991 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.232118 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.232231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.232308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.232475 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.232555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.232737 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.232831 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.232972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] identity.caching = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.233050 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.233211 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] identity.caching = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.233289 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.233456 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.233541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.233691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.233770 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.233926 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.234004 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.234164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.234251 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.234391 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.234470 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.234625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.234710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.234861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.234946 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.235105 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.235183 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.235334 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.235413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.235563 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.235650 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.235793 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.235871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.236025 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.236104 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.236265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.236352 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.236510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.236626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.236780 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.236871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.237024 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.237110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.237279 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.237357 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.237512 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.237591 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.237747 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.237827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.237975 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.238054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.238210 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.238289 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.238437 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.238517 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.238665 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.238743 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.238901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.238978 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.239137 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.239216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.239374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.239452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.239600 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.239679 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.239842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.239921 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.240083 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.240160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.240315 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.240394 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.240548 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.240654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.240813 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.240890 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.241069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.241155 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.241272 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.241353 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.241498 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.241576 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.241728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.241806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.241966 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.242046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.242202 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.242279 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.242436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.242516 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.242672 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.242750 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.242911 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.242989 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.243153 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.243230 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.243381 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.243460 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.243609 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.243687 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.243839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.243918 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.244085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.244165 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.244319 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.244398 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.244552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.244643 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.244808 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.244889 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.245036 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.user = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.245115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.245265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.user = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.245342 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.245506 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.245586 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.245735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.245813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.245981 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.246060 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.246225 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.246304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.246466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.246543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.246700 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.246779 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.246930 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.247010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.247722 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.247815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.247991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.248125 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.248234 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.248376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.248455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.248651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.248735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.248889 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.248981 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.249140 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.249220 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.249389 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.249467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.249636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.249724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.249867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.249946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.250111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.250190 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.250340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.250427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.250572 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.250649 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.250804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.250882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.251037 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.251123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.251274 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.251353 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.251510 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.251589 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.251742 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.251834 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.251992 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.252070 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.252219 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.252297 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.252460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.252634 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.252711 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.252782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.252940 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.253095 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.253179 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.253257 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.253399 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.253478 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.253635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.253712 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.253866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.253945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.254100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.254178 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.254330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.254408 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] resource.caching = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.254569 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.254649 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.254811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] resource.caching = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.254890 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.255050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.255127 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.255297 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.255383 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.255541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.255619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.255772 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.255849 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.256018 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.256098 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.256251 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.256329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.256487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.256565 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.256743 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] role.caching = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.256824 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.256980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.257058 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] role.driver = sql {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.257207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.257285 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] role.caching = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.257453 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.257540 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] role.driver = sql {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.257682 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.257761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.257929 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.258006 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.258173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.258252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.258417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.258500 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.258663 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.258740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.258904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.258982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.259148 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.259225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.259375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.259453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.259618 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.259698 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.259847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.259924 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.260093 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.260172 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.260335 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.260413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.260620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.260752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.260832 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.260989 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.261223 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.261302 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.261464 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.261542 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.261705 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.261791 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.261951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.262030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.262185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.262264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.262432 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.262518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.262698 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.262783 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.262957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.263121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.263200 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.263378 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.263458 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.263618 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.263696 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.263853 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.263932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.264087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.264167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.264319 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.264398 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.264556 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.264680 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.264814 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.264895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.265043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.265122 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.265278 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] token.caching = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.265357 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.265512 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.265591 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.265740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.265818 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.265979 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] token.caching = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.266057 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.266216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.266295 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.266457 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.266535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.266692 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.266769 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.266931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.267049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.267173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.267279 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.267413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.267518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.267649 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.267755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.267888 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.267995 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.268123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.268229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.268367 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.268473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.268627 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.268737 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.268867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.268971 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.269113 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.269220 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.269362 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.269467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.269607 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.269713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] database.connection = **** {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.269856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.269975 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.270108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.270219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.270343 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.270449 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] database.connection = **** {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.270582 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.270705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.270867 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.270980 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.271107 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.271220 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.271350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.271456 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.271587 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.271768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.271877 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.271959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.272033 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.272105 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.272272 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.272360 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.272504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.272609 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.272765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.272845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.272998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.273085 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.273232 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.273311 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.273458 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.273536 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.273696 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.273777 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.273930 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.274010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.274083 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80519) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 22:26:00.274155 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.274384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.274620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:26:00.274791 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 22:26:00.305312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.305436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.305537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.305615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.305752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.305838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.314790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.315169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.342375 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.342604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.343535 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.343766 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.486288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.486922 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.534617 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.534851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.534939 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.535133 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.599113 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.599338 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.599459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.599570 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.599694 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.599796 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.599921 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.600006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.600146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.600228 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.600366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.600659 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.625406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.625527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.625635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.625764 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.625848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.625992 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.626075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.626216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.626299 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.626438 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.626522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.626666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.626751 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.626870 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.637670 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.638014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.650418 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.650603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.650704 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.650839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.650923 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.651054 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.651136 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.651266 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.651348 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.651474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.651557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.651683 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80519) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.651767 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.651896 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:26:00.670313 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.670520 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 22:26:00.670831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.670917 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.671035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 22:26:00.671382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 22:26:00.671553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.671636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.671809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 22:26:00.671891 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.672059 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.672149 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.672310 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.672391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.672567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.672853 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.683223 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.683496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.683609 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.683758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.683885 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.684032 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.684153 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.684429 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.692832 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80519) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.693470 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.696072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80519) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.696347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.696646 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.696767 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.696926 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.697046 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.697204 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.697328 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.697459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.697611 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.697721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.697895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.697986 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.698158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.698238 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.698425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.698505 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.698696 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.698777 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.698964 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.699235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.699502 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.725431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.726598 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.727019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80519) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.727294 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.728208 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.728484 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.733076 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80519) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.733468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.733748 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.734337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.734730 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.735013 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.741009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80519) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.741250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.742302 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.742584 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.745679 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80519) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.745961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.746239 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.746518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.746788 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.747604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.747720 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.747894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.748021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.748174 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.748263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.748452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.748754 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.749031 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.749301 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.749568 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.766949 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.767220 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.768511 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.768812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.771793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80519) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.773423 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.775397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.775787 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.777075 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.777471 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.781511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80519) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.782620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.785083 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80519) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.786224 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.788713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.789008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.789291 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.789570 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.789841 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.789925 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.790114 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.790208 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.790476 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.790762 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.791040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.791319 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.810474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80519) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.810762 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.811031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.811300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.811766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.812049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.812316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.812610 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.819132 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.820462 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.820827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.821103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.822190 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.822465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.824272 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80519) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.824549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.824851 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.825201 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.825481 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.825868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.826151 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.826435 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.826716 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.827003 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.838667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80519) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.838944 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.839231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.839509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.840256 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.840539 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.840899 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.841133 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.841265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.841422 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.841705 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.842826 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.843119 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.843403 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.859981 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80519) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.861816 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.863238 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80519) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.865109 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.866513 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.868215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.868385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.869859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80519) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.870134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.870231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.870416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.871770 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.872606 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.872896 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.878445 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.880041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80519) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.880410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.882037 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.883260 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.883547 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.883933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.884208 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.885283 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.885573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.885957 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.886236 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.893270 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80519) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.893558 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.893834 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.895481 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.895767 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.896049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.901629 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80519) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.901904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.902178 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.902447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.902711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.903080 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.903417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.903686 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.903955 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.904040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:26:00.904234 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.904318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.904503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.904624 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.904790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.904873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.905051 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.905134 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.905318 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80519) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.905434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.905728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.906030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.906312 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.906595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.906869 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.907131 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.907398 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.907669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:26:00.944259 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55978f0979c0 pid: 80519 (default app) Jun 23 22:26:00.946873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55978f0979c0 pid: 80518 (default app) Jun 23 22:26:00.947643 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ddd62-5f12-4df2-b6a0-08b5c8d3f239 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:00.947937 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ddd62-5f12-4df2-b6a0-08b5c8d3f239 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:00.948215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ddd62-5f12-4df2-b6a0-08b5c8d3f239 None None] PATH_INFO: `/v3` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:00.952527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Thu Jun 23 22:26:00 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:09.486759 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c41db-4f5a-43ac-bb6f-671fccd978f5 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:09.487406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c41db-4f5a-43ac-bb6f-671fccd978f5 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:09.487406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c41db-4f5a-43ac-bb6f-671fccd978f5 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:09.491642 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:09.497389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f60ec5-fcda-4560-ac1b-cb94938886d3 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:09.502717 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f60ec5-fcda-4560-ac1b-cb94938886d3 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:09.502717 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f60ec5-fcda-4560-ac1b-cb94938886d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:09.567432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: INFO dbcounter [None req-a3f60ec5-fcda-4560-ac1b-cb94938886d3 None None] Registered counter for database keystone Jun 23 22:26:09.588307 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG oslo_db.sqlalchemy.engines [None req-a3f60ec5-fcda-4560-ac1b-cb94938886d3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80519) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 22:26:09.595071 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writer thread running {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 22:26:09.769805 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG passlib.handlers.bcrypt [None req-a3f60ec5-fcda-4560-ac1b-cb94938886d3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80519) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 22:26:09.770229 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG passlib.handlers.bcrypt [None req-a3f60ec5-fcda-4560-ac1b-cb94938886d3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80519) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 22:26:09.824206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-a3f60ec5-fcda-4560-ac1b-cb94938886d3 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:09.826374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-a3f60ec5-fcda-4560-ac1b-cb94938886d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:09.946363 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 23 22:26:10.112565 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c41db-4f5a-43ac-bb6f-671fccd978f5 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:10.182503 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: INFO dbcounter [None req-824c41db-4f5a-43ac-bb6f-671fccd978f5 None None] Registered counter for database keystone Jun 23 22:26:10.202458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG oslo_db.sqlalchemy.engines [None req-824c41db-4f5a-43ac-bb6f-671fccd978f5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80518) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 22:26:10.208625 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writer thread running {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 22:26:10.329869 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c41db-4f5a-43ac-bb6f-671fccd978f5 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:10.334182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a6cc0-ac9f-448d-b260-84ad23a8f11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:10.338138 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a6cc0-ac9f-448d-b260-84ad23a8f11f admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:10.338138 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a6cc0-ac9f-448d-b260-84ad23a8f11f admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:10.338138 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a6cc0-ac9f-448d-b260-84ad23a8f11f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:10.343712 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a6cc0-ac9f-448d-b260-84ad23a8f11f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:10.395673 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a45a6cc0-ac9f-448d-b260-84ad23a8f11f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 22:26:10.395968 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a6cc0-ac9f-448d-b260-84ad23a8f11f admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:10.405751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Thu Jun 23 22:26:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:26:10.411734 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f60ec5-fcda-4560-ac1b-cb94938886d3 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:10.426345 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f60ec5-fcda-4560-ac1b-cb94938886d3 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:10.431506 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94459d2e-9d81-43d6-8aaa-ca34f3d4da6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:10.432656 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94459d2e-9d81-43d6-8aaa-ca34f3d4da6d admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:10.432949 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94459d2e-9d81-43d6-8aaa-ca34f3d4da6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:10.433228 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94459d2e-9d81-43d6-8aaa-ca34f3d4da6d admin admin] PATH_INFO: `/v3/projects/3f0d9b0cc6e44015a1eedda4627f582b` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:10.435413 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94459d2e-9d81-43d6-8aaa-ca34f3d4da6d admin admin] RBAC: Authorizing `identity:get_project(project_id=3f0d9b0cc6e44015a1eedda4627f582b)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:10.488521 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: WARNING keystone.common.rbac_enforcer.enforcer [None req-94459d2e-9d81-43d6-8aaa-ca34f3d4da6d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 22:26:10.488888 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94459d2e-9d81-43d6-8aaa-ca34f3d4da6d admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:10.493424 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Thu Jun 23 22:26:10 2022] GET /identity/v3/projects/3f0d9b0cc6e44015a1eedda4627f582b => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:12.541655 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986547aa-982a-4e34-8c4c-b373b037f3f2 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:12.542163 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986547aa-982a-4e34-8c4c-b373b037f3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:12.542163 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986547aa-982a-4e34-8c4c-b373b037f3f2 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:12.547338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:26:12.552783 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb199141-1e29-462d-8813-b6966f24e387 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:12.553245 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb199141-1e29-462d-8813-b6966f24e387 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:12.553361 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb199141-1e29-462d-8813-b6966f24e387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:12.572122 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-fb199141-1e29-462d-8813-b6966f24e387 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:12.573685 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-fb199141-1e29-462d-8813-b6966f24e387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:12.581840 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 23 22:26:12.603887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986547aa-982a-4e34-8c4c-b373b037f3f2 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:12.616631 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986547aa-982a-4e34-8c4c-b373b037f3f2 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:12.620805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81416956-2316-4ea3-af16-a9bf08fdf863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:12.621493 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81416956-2316-4ea3-af16-a9bf08fdf863 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:12.621768 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81416956-2316-4ea3-af16-a9bf08fdf863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:12.622053 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81416956-2316-4ea3-af16-a9bf08fdf863 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:12.629762 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81416956-2316-4ea3-af16-a9bf08fdf863 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:12.631533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81416956-2316-4ea3-af16-a9bf08fdf863 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:12.640241 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Thu Jun 23 22:26:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:26:12.645452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb199141-1e29-462d-8813-b6966f24e387 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:12.655133 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb199141-1e29-462d-8813-b6966f24e387 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:12.659537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79b1ed-73f5-4f7c-8da0-868fc8c34ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:12.660400 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79b1ed-73f5-4f7c-8da0-868fc8c34ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:12.660729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79b1ed-73f5-4f7c-8da0-868fc8c34ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:12.661014 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79b1ed-73f5-4f7c-8da0-868fc8c34ea5 admin admin] PATH_INFO: `/v3/users/091e3a1287aa48ddab1a4bfa5166599f` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:12.662407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79b1ed-73f5-4f7c-8da0-868fc8c34ea5 admin admin] RBAC: Authorizing `identity:get_user(user_id=091e3a1287aa48ddab1a4bfa5166599f)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:12.664253 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79b1ed-73f5-4f7c-8da0-868fc8c34ea5 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:12.665851 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Thu Jun 23 22:26:12 2022] GET /identity/v3/users/091e3a1287aa48ddab1a4bfa5166599f => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:14.728612 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f76c7a-e761-40e7-b8ff-c0a2476280ff None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:14.729111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f76c7a-e761-40e7-b8ff-c0a2476280ff None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:14.729111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f76c7a-e761-40e7-b8ff-c0a2476280ff None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:14.730489 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:14.735455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46baf8b1-4a96-4052-bfec-52610e93e59b None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:14.735882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46baf8b1-4a96-4052-bfec-52610e93e59b None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:14.736031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46baf8b1-4a96-4052-bfec-52610e93e59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:14.753311 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-46baf8b1-4a96-4052-bfec-52610e93e59b None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:14.754508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-46baf8b1-4a96-4052-bfec-52610e93e59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:14.785049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570228f8-c14f-4a08-a248-c0fcb938361e None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:14.785339 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570228f8-c14f-4a08-a248-c0fcb938361e None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:14.785622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570228f8-c14f-4a08-a248-c0fcb938361e None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:14.786957 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:26:14.792057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0db6a-ad75-493c-b3af-69ce7f970397 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:14.792668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0db6a-ad75-493c-b3af-69ce7f970397 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:14.792982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0db6a-ad75-493c-b3af-69ce7f970397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:14.832568 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG passlib.handlers.bcrypt [None req-5dd0db6a-ad75-493c-b3af-69ce7f970397 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80518) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 22:26:14.833237 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG passlib.handlers.bcrypt [None req-5dd0db6a-ad75-493c-b3af-69ce7f970397 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80518) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 22:26:14.837758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 6/12] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:14.860255 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46baf8b1-4a96-4052-bfec-52610e93e59b None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:14.878853 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-5dd0db6a-ad75-493c-b3af-69ce7f970397 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:14.880453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-5dd0db6a-ad75-493c-b3af-69ce7f970397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:14.883510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:14.963206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46baf8b1-4a96-4052-bfec-52610e93e59b None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:14.983523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 7/13] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:26:14.998250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ccd22-16b8-4b29-af58-3d8f545a89ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:14.999064 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ccd22-16b8-4b29-af58-3d8f545a89ed None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:14.999354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ccd22-16b8-4b29-af58-3d8f545a89ed None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:14.999639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ccd22-16b8-4b29-af58-3d8f545a89ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:15.006374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0db6a-ad75-493c-b3af-69ce7f970397 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:15.011231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645ccd22-16b8-4b29-af58-3d8f545a89ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:15.023048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-645ccd22-16b8-4b29-af58-3d8f545a89ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:15.023048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:15.023048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:15.023048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:15.023048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:15.023048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:15.023048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:15.023048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:15.023048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:15.023048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:15.023048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:15.023048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:15.023048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:15.023048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:15.023048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:15.023048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:15.023972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:15.023972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:15.023972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:15.023972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:15.023972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:15.023972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:15.023972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:15.023972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:15.023972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:15.023972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:15.023972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:15.023972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:15.023972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:15.023972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:15.023972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:15.023972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:15.024822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:15.024822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:15.024822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:15.024822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:15.024822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:15.024822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:15.024822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:15.024822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:15.024822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:15.024822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:15.024822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:15.024822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:26:15.024822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 7/14] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:26:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:26:15.029984 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ccd22-16b8-4b29-af58-3d8f545a89ed None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:15.109437 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0db6a-ad75-493c-b3af-69ce7f970397 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:15.111719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:15.113744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:15.132190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ccd22-16b8-4b29-af58-3d8f545a89ed None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:15.146195 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df832b9-3f4b-4e06-921c-d7d228ff8dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:15.147088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df832b9-3f4b-4e06-921c-d7d228ff8dad None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:15.147375 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df832b9-3f4b-4e06-921c-d7d228ff8dad None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:15.147655 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df832b9-3f4b-4e06-921c-d7d228ff8dad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:15.165552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e3ba7-a8a8-4d71-9554-1edc38fc0479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:15.166226 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e3ba7-a8a8-4d71-9554-1edc38fc0479 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:15.166519 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e3ba7-a8a8-4d71-9554-1edc38fc0479 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:15.166800 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e3ba7-a8a8-4d71-9554-1edc38fc0479 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:15.167583 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e3ba7-a8a8-4d71-9554-1edc38fc0479 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:15.167895 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e3ba7-a8a8-4d71-9554-1edc38fc0479 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:15.170238 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e3ba7-a8a8-4d71-9554-1edc38fc0479 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:15.182671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 8/15] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:15.194320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e3ba7-a8a8-4d71-9554-1edc38fc0479 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:15.214319 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df832b9-3f4b-4e06-921c-d7d228ff8dad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:15.215876 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df832b9-3f4b-4e06-921c-d7d228ff8dad None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:15.289013 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e3ba7-a8a8-4d71-9554-1edc38fc0479 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:15.314126 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 8/16] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 22:26:15.319729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df832b9-3f4b-4e06-921c-d7d228ff8dad None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:15.322410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:15.325402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a581c2-a270-4987-9321-c0815e19e268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:15.326282 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a581c2-a270-4987-9321-c0815e19e268 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:15.326574 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a581c2-a270-4987-9321-c0815e19e268 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:15.326855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a581c2-a270-4987-9321-c0815e19e268 None admin] PATH_INFO: `/v3/users/091e3a1287aa48ddab1a4bfa5166599f` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:15.328086 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a581c2-a270-4987-9321-c0815e19e268 None admin] RBAC: Authorizing `identity:get_user(user_id=091e3a1287aa48ddab1a4bfa5166599f)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:15.329670 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a581c2-a270-4987-9321-c0815e19e268 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:15.335689 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 9/17] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:15 2022] GET /identity/v3/users/091e3a1287aa48ddab1a4bfa5166599f => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:15.336644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a581c2-a270-4987-9321-c0815e19e268 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:15.350975 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:15.416857 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df832b9-3f4b-4e06-921c-d7d228ff8dad None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:15.438267 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a581c2-a270-4987-9321-c0815e19e268 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:15.451012 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f2004-6704-45a3-883b-a069e0103e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:15.451804 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f2004-6704-45a3-883b-a069e0103e39 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:15.452092 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f2004-6704-45a3-883b-a069e0103e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:15.452370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f2004-6704-45a3-883b-a069e0103e39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:15.471343 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b914ad9-4857-4022-8796-3999df4febb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:15.472134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b914ad9-4857-4022-8796-3999df4febb7 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:15.472303 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f2004-6704-45a3-883b-a069e0103e39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:15.472411 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b914ad9-4857-4022-8796-3999df4febb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:15.472716 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b914ad9-4857-4022-8796-3999df4febb7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:15.473815 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f2004-6704-45a3-883b-a069e0103e39 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:15.473901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b914ad9-4857-4022-8796-3999df4febb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:15.475296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b914ad9-4857-4022-8796-3999df4febb7 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:15.477096 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 10/18] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:15.483387 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b914ad9-4857-4022-8796-3999df4febb7 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:15.494748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-1f5f2004-6704-45a3-883b-a069e0103e39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:15.494748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:15.494748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:15.494748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:15.494748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:15.494748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:15.494748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:15.494748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:15.494748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:15.494748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:15.494748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:15.494748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:15.494748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:26:15.494748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:26:15.494748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:26:15.495546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:26:15.495546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:15.495546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:15.495546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:15.495546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:15.495546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:15.495546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:15.495546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:15.495546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:15.495546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:15.495546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:15.495546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:15.495546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:15.495546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:15.495546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:15.495546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:15.495546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:15.496470 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:26:15.496470 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:26:15.496470 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:15.496470 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:26:15.496470 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 9/19] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:26:15.501537 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f2004-6704-45a3-883b-a069e0103e39 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:15.557304 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:15.569108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:15.587398 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b914ad9-4857-4022-8796-3999df4febb7 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:15.600937 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f2004-6704-45a3-883b-a069e0103e39 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:15.622004 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1eb2f-456f-4fc7-8c5e-59498f631253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:15.622661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1eb2f-456f-4fc7-8c5e-59498f631253 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:15.622946 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1eb2f-456f-4fc7-8c5e-59498f631253 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:15.623233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1eb2f-456f-4fc7-8c5e-59498f631253 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:15.624058 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1eb2f-456f-4fc7-8c5e-59498f631253 None admin] RBAC: Adding query filter params (role.id=6ff7708b59984b9287cc81d46a57b4b4, scope.domain.id=default, user.id=091e3a1287aa48ddab1a4bfa5166599f) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:15.624372 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1eb2f-456f-4fc7-8c5e-59498f631253 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:15.626195 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1eb2f-456f-4fc7-8c5e-59498f631253 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:15.634315 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 11/20] 10.222.0.33 () {56 vars in 1322 bytes} [Thu Jun 23 22:26:15 2022] GET /identity/v3/role_assignments?user.id=091e3a1287aa48ddab1a4bfa5166599f&scope.domain.id=default&role.id=6ff7708b59984b9287cc81d46a57b4b4 => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:15.634811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e217942-4db8-4050-a02e-b04fad0ce06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:15.635470 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e217942-4db8-4050-a02e-b04fad0ce06d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:15.635757 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e217942-4db8-4050-a02e-b04fad0ce06d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:15.636036 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e217942-4db8-4050-a02e-b04fad0ce06d None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:15.636825 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e217942-4db8-4050-a02e-b04fad0ce06d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:15.637139 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e217942-4db8-4050-a02e-b04fad0ce06d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:15.638539 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e217942-4db8-4050-a02e-b04fad0ce06d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:15.658666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 10/21] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:17.962381 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ce96c-1294-4fd2-983d-ed1ee9999486 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:17.962882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ce96c-1294-4fd2-983d-ed1ee9999486 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:17.962957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ce96c-1294-4fd2-983d-ed1ee9999486 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:17.964477 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 12/22] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:17.970627 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be215c1-df88-44f4-bf18-8335b1dd0ab2 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:17.971523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be215c1-df88-44f4-bf18-8335b1dd0ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:17.972005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be215c1-df88-44f4-bf18-8335b1dd0ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:18.001985 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-6be215c1-df88-44f4-bf18-8335b1dd0ab2 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:18.003528 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6be215c1-df88-44f4-bf18-8335b1dd0ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:18.025965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d2fc1-20d4-4645-a668-e54a93e1dee3 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:18.026278 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d2fc1-20d4-4645-a668-e54a93e1dee3 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:18.026547 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d2fc1-20d4-4645-a668-e54a93e1dee3 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:18.029151 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 13/23] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:26:18.034235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9ddfc-f2c9-4b83-9ed0-7b9be0e5c3c8 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:18.034531 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9ddfc-f2c9-4b83-9ed0-7b9be0e5c3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:18.034817 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9ddfc-f2c9-4b83-9ed0-7b9be0e5c3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:18.068667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-13d9ddfc-f2c9-4b83-9ed0-7b9be0e5c3c8 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:18.069874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-13d9ddfc-f2c9-4b83-9ed0-7b9be0e5c3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:18.172509 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 11/24] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:26:18.189520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15217848-2742-4e44-bbb4-24d2ad0d7506 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:18.189811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15217848-2742-4e44-bbb4-24d2ad0d7506 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:18.190110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15217848-2742-4e44-bbb4-24d2ad0d7506 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:18.191479 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 12/25] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:26:18.203463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba34a3-2dd8-4f74-ac07-98c7eb365519 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:18.203765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba34a3-2dd8-4f74-ac07-98c7eb365519 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:18.204058 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba34a3-2dd8-4f74-ac07-98c7eb365519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:18.219150 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 14/26] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:26:18.223140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9ddfc-f2c9-4b83-9ed0-7b9be0e5c3c8 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:18.241235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-56ba34a3-2dd8-4f74-ac07-98c7eb365519 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:18.242438 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-56ba34a3-2dd8-4f74-ac07-98c7eb365519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:18.341035 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:18.368917 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:18.384757 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 13/27] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:26:18.385623 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba34a3-2dd8-4f74-ac07-98c7eb365519 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:18.420212 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9ddfc-f2c9-4b83-9ed0-7b9be0e5c3c8 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:18.493899 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba34a3-2dd8-4f74-ac07-98c7eb365519 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:18.500372 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28741eb-f3e1-47b4-84ea-991452376725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:18.501207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28741eb-f3e1-47b4-84ea-991452376725 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:18.501492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28741eb-f3e1-47b4-84ea-991452376725 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:18.501770 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28741eb-f3e1-47b4-84ea-991452376725 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:18.520669 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28741eb-f3e1-47b4-84ea-991452376725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:18.533844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-f28741eb-f3e1-47b4-84ea-991452376725 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:18.533844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:18.533844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:18.533844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:18.533844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:18.533844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:18.533844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:18.533844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:18.533844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:18.533844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:18.533844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:18.533844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:18.533844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:18.533844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:18.533844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:18.533844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:18.534721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:18.534721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:18.534721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:18.534721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:18.534721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:18.534721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:18.534721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:18.534721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:18.534721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:18.534721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:18.534721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:18.534721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:18.534721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:18.534721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:18.534721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:18.534721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:18.535467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:18.535467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:18.535467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:18.535467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:18.535467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:18.535467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:18.535467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:18.535467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:18.535467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:18.535467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:18.535467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:18.535467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:26:18.537855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 15/28] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:26:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:26:18.538820 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28741eb-f3e1-47b4-84ea-991452376725 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:18.553300 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba06a2-9cab-4845-90fb-ec37893ad4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:18.554307 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba06a2-9cab-4845-90fb-ec37893ad4ff None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:18.554782 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba06a2-9cab-4845-90fb-ec37893ad4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:18.555227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba06a2-9cab-4845-90fb-ec37893ad4ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:18.597479 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba06a2-9cab-4845-90fb-ec37893ad4ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:18.599229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba06a2-9cab-4845-90fb-ec37893ad4ff None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:18.637363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-faba06a2-9cab-4845-90fb-ec37893ad4ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:18.637363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:18.637363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:18.637363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:18.637363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:18.637363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:18.637363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:18.637363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:18.637363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:18.637363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:18.637363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:18.637363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:18.637363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:26:18.637363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:26:18.637363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:26:18.638271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:26:18.638271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:18.638271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:18.638271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:18.638271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:18.638271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:18.638271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:18.638271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:18.638271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:18.638271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:18.638271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:18.638271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:18.638271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:18.638271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:18.638271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:18.638271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:18.638271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:18.639056 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:26:18.639056 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:26:18.639056 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:18.639056 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:26:18.645166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 14/29] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:18.647906 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba06a2-9cab-4845-90fb-ec37893ad4ff None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:18.689469 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28741eb-f3e1-47b4-84ea-991452376725 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:18.708951 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:18.748639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf22f74-28d8-4f50-aa58-9ef8e563b38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:18.749958 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf22f74-28d8-4f50-aa58-9ef8e563b38d None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:18.750679 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf22f74-28d8-4f50-aa58-9ef8e563b38d None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:18.751157 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf22f74-28d8-4f50-aa58-9ef8e563b38d None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:18.752264 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf22f74-28d8-4f50-aa58-9ef8e563b38d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:18.755503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf22f74-28d8-4f50-aa58-9ef8e563b38d None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:18.759513 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:18.790382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba06a2-9cab-4845-90fb-ec37893ad4ff None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:18.827419 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83028276-060e-4e8b-aa01-15047a82282d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:18.828092 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83028276-060e-4e8b-aa01-15047a82282d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:18.828386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83028276-060e-4e8b-aa01-15047a82282d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:18.828706 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83028276-060e-4e8b-aa01-15047a82282d None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:18.829492 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83028276-060e-4e8b-aa01-15047a82282d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:18.829789 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83028276-060e-4e8b-aa01-15047a82282d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:18.831173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83028276-060e-4e8b-aa01-15047a82282d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:18.847324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 15/30] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:18.847324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83028276-060e-4e8b-aa01-15047a82282d None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:18.948894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:18.959561 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83028276-060e-4e8b-aa01-15047a82282d None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:18.985836 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 16/31] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Jun 23 22:26:18 2022] POST /identity/v3/roles => generated 219 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:26:18.987489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370b2f3-8d30-4466-8627-7010fecd491d None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:18.987795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370b2f3-8d30-4466-8627-7010fecd491d None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:18.988144 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370b2f3-8d30-4466-8627-7010fecd491d None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:18.989856 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 17/32] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:18.990645 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370b2f3-8d30-4466-8627-7010fecd491d None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:19.003799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314087b-40db-445e-99db-5b694969c24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:19.005185 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314087b-40db-445e-99db-5b694969c24c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:19.005690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314087b-40db-445e-99db-5b694969c24c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:19.006177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314087b-40db-445e-99db-5b694969c24c None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:19.007124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314087b-40db-445e-99db-5b694969c24c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:19.007635 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314087b-40db-445e-99db-5b694969c24c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:19.009275 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314087b-40db-445e-99db-5b694969c24c None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:19.022867 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 16/33] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:19.024797 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b55d4a-c09b-4ba5-a9fb-c4349b4271ad None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:19.025313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b55d4a-c09b-4ba5-a9fb-c4349b4271ad None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:19.025789 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b55d4a-c09b-4ba5-a9fb-c4349b4271ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:19.046158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-33b55d4a-c09b-4ba5-a9fb-c4349b4271ad None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:19.047820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-33b55d4a-c09b-4ba5-a9fb-c4349b4271ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:19.127421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370b2f3-8d30-4466-8627-7010fecd491d None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:19.132596 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 17/34] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:19.133700 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b55d4a-c09b-4ba5-a9fb-c4349b4271ad None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:19.171688 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84480493-7b7e-4b5a-8641-f6a07da92b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:19.172684 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84480493-7b7e-4b5a-8641-f6a07da92b71 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:19.172980 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84480493-7b7e-4b5a-8641-f6a07da92b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:19.173264 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84480493-7b7e-4b5a-8641-f6a07da92b71 None admin] PATH_INFO: `/v3/users/091e3a1287aa48ddab1a4bfa5166599f` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:19.176750 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84480493-7b7e-4b5a-8641-f6a07da92b71 None admin] RBAC: Authorizing `identity:get_user(user_id=091e3a1287aa48ddab1a4bfa5166599f)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:19.177075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84480493-7b7e-4b5a-8641-f6a07da92b71 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:19.180273 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 18/35] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:18 2022] GET /identity/v3/users/091e3a1287aa48ddab1a4bfa5166599f => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:19.180273 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84480493-7b7e-4b5a-8641-f6a07da92b71 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:19.248337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:19.272471 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:19.274001 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b55d4a-c09b-4ba5-a9fb-c4349b4271ad None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:19.323822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d4c92-8343-423b-88e9-bb57490cdee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:19.325141 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d4c92-8343-423b-88e9-bb57490cdee8 None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:19.325430 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d4c92-8343-423b-88e9-bb57490cdee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:19.325716 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d4c92-8343-423b-88e9-bb57490cdee8 None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:19.326743 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0d4c92-8343-423b-88e9-bb57490cdee8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:19.328281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0d4c92-8343-423b-88e9-bb57490cdee8 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:19.330879 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84480493-7b7e-4b5a-8641-f6a07da92b71 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:19.373694 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac381110-6a8e-4ce0-b17b-304f9342bb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:19.374687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac381110-6a8e-4ce0-b17b-304f9342bb4c None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:19.374919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac381110-6a8e-4ce0-b17b-304f9342bb4c None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:19.375201 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac381110-6a8e-4ce0-b17b-304f9342bb4c None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:19.375968 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac381110-6a8e-4ce0-b17b-304f9342bb4c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:19.377460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac381110-6a8e-4ce0-b17b-304f9342bb4c None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:19.483934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 19/36] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Jun 23 22:26:19 2022] POST /identity/v3/roles => generated 225 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:26:19.484807 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac381110-6a8e-4ce0-b17b-304f9342bb4c None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:19.557945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 18/37] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 22:26:19 2022] POST /identity/v3/projects => generated 301 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:26:19.560530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfc48d-9ac1-4061-8e96-44f2e2606b93 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:19.561054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfc48d-9ac1-4061-8e96-44f2e2606b93 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:19.561535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfc48d-9ac1-4061-8e96-44f2e2606b93 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:19.563458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 19/38] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:19.574272 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d320b3-5b45-49c0-b81f-2f2bbe026756 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:19.574951 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d320b3-5b45-49c0-b81f-2f2bbe026756 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:19.575443 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d320b3-5b45-49c0-b81f-2f2bbe026756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:19.622569 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-e3d320b3-5b45-49c0-b81f-2f2bbe026756 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:19.624825 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e3d320b3-5b45-49c0-b81f-2f2bbe026756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:19.722962 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac381110-6a8e-4ce0-b17b-304f9342bb4c None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:19.753698 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 20/39] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:19.754534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:19.769892 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6697a9-653c-4b9c-b923-c985d21a5c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:19.770712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6697a9-653c-4b9c-b923-c985d21a5c47 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:19.771008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6697a9-653c-4b9c-b923-c985d21a5c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:19.771298 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6697a9-653c-4b9c-b923-c985d21a5c47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:19.773351 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6697a9-653c-4b9c-b923-c985d21a5c47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:19.774840 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6697a9-653c-4b9c-b923-c985d21a5c47 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:19.776530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 20/40] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:19.784551 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d320b3-5b45-49c0-b81f-2f2bbe026756 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:19.800813 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6697a9-653c-4b9c-b923-c985d21a5c47 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:19.865033 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:19.905351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d320b3-5b45-49c0-b81f-2f2bbe026756 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:19.914100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6697a9-653c-4b9c-b923-c985d21a5c47 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:19.939971 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26892d-0fe5-4838-b05c-4c98c97fd9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:19.940880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26892d-0fe5-4838-b05c-4c98c97fd9f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:19.941165 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26892d-0fe5-4838-b05c-4c98c97fd9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:19.941450 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26892d-0fe5-4838-b05c-4c98c97fd9f2 None admin] PATH_INFO: `/v3/domains/default/users/091e3a1287aa48ddab1a4bfa5166599f/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:19.943700 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26892d-0fe5-4838-b05c-4c98c97fd9f2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=091e3a1287aa48ddab1a4bfa5166599f, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:19.945314 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26892d-0fe5-4838-b05c-4c98c97fd9f2 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:19.948016 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92efce-184e-4a46-a67b-7c17fe3b6849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:19.948725 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92efce-184e-4a46-a67b-7c17fe3b6849 None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:19.949661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92efce-184e-4a46-a67b-7c17fe3b6849 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:19.949947 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92efce-184e-4a46-a67b-7c17fe3b6849 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:19.950710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92efce-184e-4a46-a67b-7c17fe3b6849 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:19.952179 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92efce-184e-4a46-a67b-7c17fe3b6849 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:20.032908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 21/41] 10.222.0.33 () {58 vars in 1361 bytes} [Thu Jun 23 22:26:19 2022] PUT /identity/v3/domains/default/users/091e3a1287aa48ddab1a4bfa5166599f/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:26:20.034197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f2051-f645-4764-8c54-dad94e6fc559 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:20.034488 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f2051-f645-4764-8c54-dad94e6fc559 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:20.034768 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f2051-f645-4764-8c54-dad94e6fc559 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:20.036038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 22/42] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:20.041813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35878584-b684-44ed-ba22-58454f92ce1e None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:20.042105 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35878584-b684-44ed-ba22-58454f92ce1e None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:20.042414 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35878584-b684-44ed-ba22-58454f92ce1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:20.061263 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-35878584-b684-44ed-ba22-58454f92ce1e None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:20.062497 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-35878584-b684-44ed-ba22-58454f92ce1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:20.063374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 21/43] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Jun 23 22:26:19 2022] POST /identity/v3/roles => generated 223 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:26:20.144873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 23/44] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:26:20.168879 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92efce-184e-4a46-a67b-7c17fe3b6849 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:20.229850 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:20.282297 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92efce-184e-4a46-a67b-7c17fe3b6849 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:20.320006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbcf496-f4f7-42a2-9f35-bd9abd8c9cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:20.320866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcf496-f4f7-42a2-9f35-bd9abd8c9cee None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:20.321165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcf496-f4f7-42a2-9f35-bd9abd8c9cee None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:20.321452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcf496-f4f7-42a2-9f35-bd9abd8c9cee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:20.322653 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbcf496-f4f7-42a2-9f35-bd9abd8c9cee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:20.324749 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbcf496-f4f7-42a2-9f35-bd9abd8c9cee None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:20.326787 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 22/45] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:20.332241 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35878584-b684-44ed-ba22-58454f92ce1e None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:20.384771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:20.434103 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35878584-b684-44ed-ba22-58454f92ce1e None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:20.466778 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68325e-f8a3-42b4-9107-f4f50e6681b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:20.467441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68325e-f8a3-42b4-9107-f4f50e6681b8 None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:20.467731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68325e-f8a3-42b4-9107-f4f50e6681b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:20.468012 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68325e-f8a3-42b4-9107-f4f50e6681b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:20.468761 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68325e-f8a3-42b4-9107-f4f50e6681b8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:20.470214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68325e-f8a3-42b4-9107-f4f50e6681b8 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:20.554086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 24/46] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 22:26:20 2022] POST /identity/v3/projects => generated 312 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:26:22.076818 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e4136-e368-4d7c-a707-b7a21ca2db7a None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:22.077333 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e4136-e368-4d7c-a707-b7a21ca2db7a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:22.077435 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e4136-e368-4d7c-a707-b7a21ca2db7a None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:22.078801 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 23/47] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:22.084137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f1971-e66a-4ca8-8ce9-e381494d146a None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:22.084547 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f1971-e66a-4ca8-8ce9-e381494d146a None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:22.084733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f1971-e66a-4ca8-8ce9-e381494d146a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:22.102015 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-f55f1971-e66a-4ca8-8ce9-e381494d146a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:22.103204 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-f55f1971-e66a-4ca8-8ce9-e381494d146a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:22.274841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 25/48] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:22.299214 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e4136-e368-4d7c-a707-b7a21ca2db7a None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:22.398658 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e4136-e368-4d7c-a707-b7a21ca2db7a None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:22.416247 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:22.433610 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807df32f-cece-4191-b447-6b2bf3f7bc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:22.434413 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807df32f-cece-4191-b447-6b2bf3f7bc2a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:22.434701 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807df32f-cece-4191-b447-6b2bf3f7bc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:22.434982 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807df32f-cece-4191-b447-6b2bf3f7bc2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:22.445096 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807df32f-cece-4191-b447-6b2bf3f7bc2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:22.453081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-807df32f-cece-4191-b447-6b2bf3f7bc2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:22.453081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:22.453081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:22.453081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:22.453081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:22.453081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:22.453081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:22.453081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:22.453081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:22.453081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:22.453081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:22.453081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:22.453081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:22.453081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:22.453081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:22.453081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:22.453925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:22.453925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:22.453925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:22.453925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:22.453925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:22.453925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:22.453925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:22.453925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:22.453925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:22.453925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:22.453925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:22.453925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:22.453925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:22.453925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:22.453925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:22.453925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:22.454666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:22.454666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:22.454666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:22.454666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:22.454666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:22.454666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:22.454666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:22.454666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:22.454666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:22.454666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:22.454666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:22.454666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:26:22.454666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 24/49] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:26:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:22.460190 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f1971-e66a-4ca8-8ce9-e381494d146a None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:22.493797 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:22.560681 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f1971-e66a-4ca8-8ce9-e381494d146a None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:22.568941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cac2a1-f958-474d-be1f-c578a3b3dc6e None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:22.569234 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cac2a1-f958-474d-be1f-c578a3b3dc6e None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:22.569519 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cac2a1-f958-474d-be1f-c578a3b3dc6e None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:22.570876 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 25/50] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:22.576377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f42b9-cc6e-4820-8711-0056ef31bd1a None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:22.576705 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f42b9-cc6e-4820-8711-0056ef31bd1a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:22.576993 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f42b9-cc6e-4820-8711-0056ef31bd1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:22.594188 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-de8f42b9-cc6e-4820-8711-0056ef31bd1a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:22.594367 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779dfea-d701-4b06-8659-54d427ff152a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:22.595010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779dfea-d701-4b06-8659-54d427ff152a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:22.595300 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779dfea-d701-4b06-8659-54d427ff152a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:22.595582 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779dfea-d701-4b06-8659-54d427ff152a None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:22.595704 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-de8f42b9-cc6e-4820-8711-0056ef31bd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:22.596340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779dfea-d701-4b06-8659-54d427ff152a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:22.596674 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779dfea-d701-4b06-8659-54d427ff152a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:22.598065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779dfea-d701-4b06-8659-54d427ff152a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:22.606550 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 26/51] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:22.611893 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779dfea-d701-4b06-8659-54d427ff152a None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:22.663187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 26/52] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:26:22.687716 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f42b9-cc6e-4820-8711-0056ef31bd1a None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:22.705114 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:22.711123 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779dfea-d701-4b06-8659-54d427ff152a None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:22.745613 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2b21d-3fec-4b43-b373-a81b3e8fff82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:22.747261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2b21d-3fec-4b43-b373-a81b3e8fff82 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:22.747551 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2b21d-3fec-4b43-b373-a81b3e8fff82 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:22.747841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2b21d-3fec-4b43-b373-a81b3e8fff82 None admin] PATH_INFO: `/v3/users/091e3a1287aa48ddab1a4bfa5166599f` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:22.749091 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b2b21d-3fec-4b43-b373-a81b3e8fff82 None admin] RBAC: Authorizing `identity:get_user(user_id=091e3a1287aa48ddab1a4bfa5166599f)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:22.750550 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b2b21d-3fec-4b43-b373-a81b3e8fff82 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:22.752343 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 27/53] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:22 2022] GET /identity/v3/users/091e3a1287aa48ddab1a4bfa5166599f => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:22.757458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2b21d-3fec-4b43-b373-a81b3e8fff82 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:22.784491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f42b9-cc6e-4820-8711-0056ef31bd1a None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:22.794407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:22.819376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f18a5-c67f-470e-8ade-a8411805206a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:22.820333 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f18a5-c67f-470e-8ade-a8411805206a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:22.820920 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f18a5-c67f-470e-8ade-a8411805206a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:22.821205 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f18a5-c67f-470e-8ade-a8411805206a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:22.822382 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f18a5-c67f-470e-8ade-a8411805206a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:22.823896 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f18a5-c67f-470e-8ade-a8411805206a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:22.825777 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 27/54] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:22.831396 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f18a5-c67f-470e-8ade-a8411805206a None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:22.857503 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2b21d-3fec-4b43-b373-a81b3e8fff82 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:22.891342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f66ec9-54a6-436c-a802-4725438a0956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:22.892136 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f66ec9-54a6-436c-a802-4725438a0956 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:22.892426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f66ec9-54a6-436c-a802-4725438a0956 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:22.892737 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f66ec9-54a6-436c-a802-4725438a0956 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:22.893942 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f66ec9-54a6-436c-a802-4725438a0956 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:22.895402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f66ec9-54a6-436c-a802-4725438a0956 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:22.897058 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 28/55] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:22.903097 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f66ec9-54a6-436c-a802-4725438a0956 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:22.933913 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:22.934957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f18a5-c67f-470e-8ade-a8411805206a None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:22.969434 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b90091-914c-4a9c-bf38-ed5b40d3ed90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:22.970100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b90091-914c-4a9c-bf38-ed5b40d3ed90 None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:22.970395 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b90091-914c-4a9c-bf38-ed5b40d3ed90 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:22.970682 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b90091-914c-4a9c-bf38-ed5b40d3ed90 None admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:22.971408 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b90091-914c-4a9c-bf38-ed5b40d3ed90 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:22.972890 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b90091-914c-4a9c-bf38-ed5b40d3ed90 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:23.000267 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f66ec9-54a6-436c-a802-4725438a0956 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:23.032948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0418e-8e59-47c2-a0b1-1e7a5c52cf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:23.033602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0418e-8e59-47c2-a0b1-1e7a5c52cf79 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:23.033891 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0418e-8e59-47c2-a0b1-1e7a5c52cf79 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:23.034174 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0418e-8e59-47c2-a0b1-1e7a5c52cf79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:23.035005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0418e-8e59-47c2-a0b1-1e7a5c52cf79 None admin] RBAC: Adding query filter params (role.id=6ff7708b59984b9287cc81d46a57b4b4, scope.domain.id=default, user.id=091e3a1287aa48ddab1a4bfa5166599f) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:23.035319 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0418e-8e59-47c2-a0b1-1e7a5c52cf79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:23.036793 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0418e-8e59-47c2-a0b1-1e7a5c52cf79 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:23.045975 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 29/56] 10.222.0.33 () {56 vars in 1322 bytes} [Thu Jun 23 22:26:22 2022] GET /identity/v3/role_assignments?user.id=091e3a1287aa48ddab1a4bfa5166599f&scope.domain.id=default&role.id=6ff7708b59984b9287cc81d46a57b4b4 => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:23.051564 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 28/57] 10.222.0.33 () {60 vars in 1140 bytes} [Thu Jun 23 22:26:22 2022] POST /identity/v3/projects => generated 298 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:26:25.146755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f96c28-3e55-491c-8dd1-2b6d4cc639c6 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:25.147273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f96c28-3e55-491c-8dd1-2b6d4cc639c6 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:25.147355 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f96c28-3e55-491c-8dd1-2b6d4cc639c6 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:25.148717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 30/58] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:25.154460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc67f4f-db46-4811-989d-02e16ed6ebcf None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:25.154859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc67f4f-db46-4811-989d-02e16ed6ebcf None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:25.155030 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc67f4f-db46-4811-989d-02e16ed6ebcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:25.172413 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-3fc67f4f-db46-4811-989d-02e16ed6ebcf None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:25.173626 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3fc67f4f-db46-4811-989d-02e16ed6ebcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:25.195532 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:25.238015 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 29/59] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:25.262911 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f96c28-3e55-491c-8dd1-2b6d4cc639c6 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:25.359028 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:25.365602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f96c28-3e55-491c-8dd1-2b6d4cc639c6 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:25.400078 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980830ba-f739-45c3-8973-e95996a5d00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:25.400837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980830ba-f739-45c3-8973-e95996a5d00c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:25.401136 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980830ba-f739-45c3-8973-e95996a5d00c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:25.401495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980830ba-f739-45c3-8973-e95996a5d00c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:25.403379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980830ba-f739-45c3-8973-e95996a5d00c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:25.404937 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980830ba-f739-45c3-8973-e95996a5d00c None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:25.406616 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 31/60] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:25.412259 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc67f4f-db46-4811-989d-02e16ed6ebcf None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:25.508584 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc67f4f-db46-4811-989d-02e16ed6ebcf None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:25.542056 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90945ac4-a86b-44eb-b5b7-ea8091c42df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:25.542758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90945ac4-a86b-44eb-b5b7-ea8091c42df1 None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:25.543046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90945ac4-a86b-44eb-b5b7-ea8091c42df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:25.543332 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90945ac4-a86b-44eb-b5b7-ea8091c42df1 None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:25.544038 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90945ac4-a86b-44eb-b5b7-ea8091c42df1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:25.545784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90945ac4-a86b-44eb-b5b7-ea8091c42df1 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:25.646905 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 30/61] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 22:26:25 2022] POST /identity/v3/users => generated 275 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:26:28.849094 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e3b91-79e1-4bed-ba46-5205e5504713 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:28.849601 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e3b91-79e1-4bed-ba46-5205e5504713 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:28.849711 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e3b91-79e1-4bed-ba46-5205e5504713 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:28.851337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 32/62] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:28.857519 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b82a6-8bc6-4414-ba94-cc7746b8da49 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:28.857962 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b82a6-8bc6-4414-ba94-cc7746b8da49 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:28.858089 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b82a6-8bc6-4414-ba94-cc7746b8da49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:28.881584 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-460b82a6-8bc6-4414-ba94-cc7746b8da49 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:28.883072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-460b82a6-8bc6-4414-ba94-cc7746b8da49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:28.893223 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:28.895958 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc2bcb-6ec9-4834-9a0e-f7c1cabd63d7 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:28.896499 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc2bcb-6ec9-4834-9a0e-f7c1cabd63d7 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:28.897062 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc2bcb-6ec9-4834-9a0e-f7c1cabd63d7 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:28.898804 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 33/63] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:28.904718 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e75e1-fc38-4ba7-967a-55adf3be1f03 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:28.906624 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e75e1-fc38-4ba7-967a-55adf3be1f03 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:28.906624 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e75e1-fc38-4ba7-967a-55adf3be1f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:28.929925 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-6c7e75e1-fc38-4ba7-967a-55adf3be1f03 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:28.934782 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6c7e75e1-fc38-4ba7-967a-55adf3be1f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:28.974523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 31/64] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:26:29.046810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 34/65] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:29.049256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b82a6-8bc6-4414-ba94-cc7746b8da49 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:29.072386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e75e1-fc38-4ba7-967a-55adf3be1f03 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:29.216560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e75e1-fc38-4ba7-967a-55adf3be1f03 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:29.228811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:29.229503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b82a6-8bc6-4414-ba94-cc7746b8da49 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:29.253741 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576b7e2-2df4-4cec-b6f5-5bf95ecc4bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:29.254890 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576b7e2-2df4-4cec-b6f5-5bf95ecc4bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:29.255419 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576b7e2-2df4-4cec-b6f5-5bf95ecc4bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:29.255898 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576b7e2-2df4-4cec-b6f5-5bf95ecc4bf6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:29.267399 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100a6a5-3e33-4784-9f75-5d07642e4624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:29.268203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100a6a5-3e33-4784-9f75-5d07642e4624 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:29.268774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100a6a5-3e33-4784-9f75-5d07642e4624 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:29.269202 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100a6a5-3e33-4784-9f75-5d07642e4624 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:29.269745 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576b7e2-2df4-4cec-b6f5-5bf95ecc4bf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:29.279538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-c576b7e2-2df4-4cec-b6f5-5bf95ecc4bf6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:29.279538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:29.279538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:29.279538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:29.279538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:29.279538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:29.279538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:29.279538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:29.279538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:29.279538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:29.279538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:29.279538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:29.279538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:29.279538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:29.279538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:29.279538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:29.280434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:29.280434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:29.280434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:29.280434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:29.280434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:29.280434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:29.280434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:29.280434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:29.280434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:29.280434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:29.280434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:29.280434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:29.280434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:29.280434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:29.280434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:29.280434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:29.281235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:29.281235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:29.281235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:29.281235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:29.281235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:29.281235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:29.281235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:29.281235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:29.281235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:29.281235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:29.281235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:29.281235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:26:29.281805 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c100a6a5-3e33-4784-9f75-5d07642e4624 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:29.283691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 35/66] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:26:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:29.289687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-c100a6a5-3e33-4784-9f75-5d07642e4624 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:29.289687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:29.289687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:29.289687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:29.289687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:29.289687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:29.289687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:29.289687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:29.289687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:29.289687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:29.289687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:29.289687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:29.289687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:29.289687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:29.289687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:29.289687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:29.290494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:29.290494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:29.290494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:29.290494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:29.290494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:29.290494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:29.290494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:29.290494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:29.290494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:29.290494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:29.290494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:29.290494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:29.290494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:29.290494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:29.290494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:29.290494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:29.291269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:29.291269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:29.291269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:29.291269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:29.291269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:29.291269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:29.291269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:29.291269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:29.291269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:29.291269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:29.291269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:29.291269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:26:29.291269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 32/67] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:26:29.291889 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100a6a5-3e33-4784-9f75-5d07642e4624 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:29.293313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8a4ad-d10d-4b42-b1f6-90f4903aa781 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:29.294070 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8a4ad-d10d-4b42-b1f6-90f4903aa781 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:29.294561 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8a4ad-d10d-4b42-b1f6-90f4903aa781 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:29.295934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:29.296934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 36/68] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:29 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:29.297912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8a4ad-d10d-4b42-b1f6-90f4903aa781 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:29.402289 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100a6a5-3e33-4784-9f75-5d07642e4624 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:29.413187 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8a4ad-d10d-4b42-b1f6-90f4903aa781 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:29.436268 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61bbf7-f3ca-4207-8301-2e6d6f0284b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:29.436977 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61bbf7-f3ca-4207-8301-2e6d6f0284b0 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:29.437269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61bbf7-f3ca-4207-8301-2e6d6f0284b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:29.437547 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61bbf7-f3ca-4207-8301-2e6d6f0284b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:29.438287 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61bbf7-f3ca-4207-8301-2e6d6f0284b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:29.438754 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61bbf7-f3ca-4207-8301-2e6d6f0284b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:29.440149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61bbf7-f3ca-4207-8301-2e6d6f0284b0 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:29.449147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 33/69] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:29.450128 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1c51c-0ee4-4e55-b698-46bb855e9477 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:29.450421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1c51c-0ee4-4e55-b698-46bb855e9477 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:29.450703 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1c51c-0ee4-4e55-b698-46bb855e9477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:29.453494 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af0705-8280-4dae-b567-ae2e3dbd20d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:29.454002 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af0705-8280-4dae-b567-ae2e3dbd20d2 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:29.454288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af0705-8280-4dae-b567-ae2e3dbd20d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:29.454564 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af0705-8280-4dae-b567-ae2e3dbd20d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:29.455321 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af0705-8280-4dae-b567-ae2e3dbd20d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:29.455632 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af0705-8280-4dae-b567-ae2e3dbd20d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:29.457053 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af0705-8280-4dae-b567-ae2e3dbd20d2 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:29.466698 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 37/70] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:26:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:29.467891 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94be2ab-1985-461b-833b-ed63f35ccfea None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:29.468186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94be2ab-1985-461b-833b-ed63f35ccfea None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:29.468472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94be2ab-1985-461b-833b-ed63f35ccfea None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:29.469734 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 38/71] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:29.472042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94be2ab-1985-461b-833b-ed63f35ccfea None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:29.472792 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-34d1c51c-0ee4-4e55-b698-46bb855e9477 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:29.473996 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-34d1c51c-0ee4-4e55-b698-46bb855e9477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:29.490426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:29.533149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:29.540271 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 34/72] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:29.541634 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a4af0-733d-4b89-b79e-d72bc0ea0e10 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:29.541928 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a4af0-733d-4b89-b79e-d72bc0ea0e10 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:29.542770 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a4af0-733d-4b89-b79e-d72bc0ea0e10 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:29.544201 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 35/73] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:29.545030 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a4af0-733d-4b89-b79e-d72bc0ea0e10 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:29.581833 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94be2ab-1985-461b-833b-ed63f35ccfea None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:29.615409 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885612e-f271-4e07-a8eb-bf3db64926e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:29.616299 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885612e-f271-4e07-a8eb-bf3db64926e2 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:29.616611 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885612e-f271-4e07-a8eb-bf3db64926e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:29.616894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885612e-f271-4e07-a8eb-bf3db64926e2 None admin] PATH_INFO: `/v3/users/091e3a1287aa48ddab1a4bfa5166599f` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:29.618106 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7885612e-f271-4e07-a8eb-bf3db64926e2 None admin] RBAC: Authorizing `identity:get_user(user_id=091e3a1287aa48ddab1a4bfa5166599f)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:29.619571 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7885612e-f271-4e07-a8eb-bf3db64926e2 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:29.621756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 39/74] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:29 2022] GET /identity/v3/users/091e3a1287aa48ddab1a4bfa5166599f => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:29.622897 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7a7ba-cf3f-4ec6-8721-47f65033cb92 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:29.623193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7a7ba-cf3f-4ec6-8721-47f65033cb92 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:29.623475 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7a7ba-cf3f-4ec6-8721-47f65033cb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:29.639758 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-e6f7a7ba-cf3f-4ec6-8721-47f65033cb92 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:29.641064 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e6f7a7ba-cf3f-4ec6-8721-47f65033cb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:29.647186 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a4af0-733d-4b89-b79e-d72bc0ea0e10 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:29.682340 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6f113-9256-40cd-ae7f-3d940cd04476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:29.683732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6f113-9256-40cd-ae7f-3d940cd04476 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:29.684018 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6f113-9256-40cd-ae7f-3d940cd04476 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:29.684291 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6f113-9256-40cd-ae7f-3d940cd04476 None admin] PATH_INFO: `/v3/users/65e4eacde0544043820a4928b1ac4083` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:29.697466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6f113-9256-40cd-ae7f-3d940cd04476 None admin] RBAC: Authorizing `identity:get_user(user_id=65e4eacde0544043820a4928b1ac4083)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:29.699991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6f113-9256-40cd-ae7f-3d940cd04476 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:29.701751 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 36/75] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:29 2022] GET /identity/v3/users/65e4eacde0544043820a4928b1ac4083 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:29.703093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e1825-e68b-4391-8320-221a45f93e7c None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:29.703394 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e1825-e68b-4391-8320-221a45f93e7c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:29.703682 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e1825-e68b-4391-8320-221a45f93e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:29.708912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 40/76] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:29.709755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7a7ba-cf3f-4ec6-8721-47f65033cb92 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:29.731468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-d39e1825-e68b-4391-8320-221a45f93e7c None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:29.732706 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-d39e1825-e68b-4391-8320-221a45f93e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:29.734383 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:29.783205 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:29.804133 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 37/77] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:29.805199 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e1825-e68b-4391-8320-221a45f93e7c None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:29.817737 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7a7ba-cf3f-4ec6-8721-47f65033cb92 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:29.852249 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623b9c4-499c-4513-9269-552f1564a2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:29.853128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623b9c4-499c-4513-9269-552f1564a2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:29.853420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623b9c4-499c-4513-9269-552f1564a2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:29.853704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623b9c4-499c-4513-9269-552f1564a2a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:29.864231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623b9c4-499c-4513-9269-552f1564a2a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:29.873282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-0623b9c4-499c-4513-9269-552f1564a2a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:26:29.873282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:29.873282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:29.873282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:29.873282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:29.873282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:29.873282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:29.873282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:29.873282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:29.873282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:29.873282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:29.873282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:29.873282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:29.873282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:29.873282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:29.873282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:29.874158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:29.874158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:29.874158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:29.874158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:29.874158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:29.874158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:29.874158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:29.874158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:29.874158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:29.874158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:29.874158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:29.874158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:29.874158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:29.874158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:29.874158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:29.874158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:29.874915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:29.874915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:29.874915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:29.874915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:29.874915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:29.874915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:29.874915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:29.874915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:29.874915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:29.874915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:29.874915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:26:29.874915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:26:29.874915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 41/78] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:26:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:29.875542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623b9c4-499c-4513-9269-552f1564a2a0 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:29.907433 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e1825-e68b-4391-8320-221a45f93e7c None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:29.941661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b28f3-597e-4e91-8e52-d7bbc75b1942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:29.942317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b28f3-597e-4e91-8e52-d7bbc75b1942 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:29.942608 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b28f3-597e-4e91-8e52-d7bbc75b1942 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:29.942889 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b28f3-597e-4e91-8e52-d7bbc75b1942 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:29.949653 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:29.974927 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623b9c4-499c-4513-9269-552f1564a2a0 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:29.994658 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:30.009036 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab79c38-3875-4c80-a090-161ba069a460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:30.009735 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab79c38-3875-4c80-a090-161ba069a460 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:30.010023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab79c38-3875-4c80-a090-161ba069a460 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:30.010305 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab79c38-3875-4c80-a090-161ba069a460 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:30.010526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0b28f3-597e-4e91-8e52-d7bbc75b1942 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:30.012037 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0b28f3-597e-4e91-8e52-d7bbc75b1942 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:30.077712 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab79c38-3875-4c80-a090-161ba069a460 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:30.079189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab79c38-3875-4c80-a090-161ba069a460 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:30.110209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 38/79] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:26:30.110209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b28f3-597e-4e91-8e52-d7bbc75b1942 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:30.151678 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:30.176692 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 42/80] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:26:30.177528 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab79c38-3875-4c80-a090-161ba069a460 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:30.199206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:30.214599 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b28f3-597e-4e91-8e52-d7bbc75b1942 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:30.248965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4170861-4790-4595-b163-3df3764d14cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:30.249697 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4170861-4790-4595-b163-3df3764d14cf None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:30.249984 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4170861-4790-4595-b163-3df3764d14cf None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:30.250261 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4170861-4790-4595-b163-3df3764d14cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:30.260369 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4170861-4790-4595-b163-3df3764d14cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:30.268432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-f4170861-4790-4595-b163-3df3764d14cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:30.268432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:30.268432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:30.268432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:30.268432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:30.268432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:30.268432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:30.268432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:30.268432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:30.268432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:30.268432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:30.268432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:30.268432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:30.268432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:30.268432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:30.268432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:30.269523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:30.269523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:30.269523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:30.269523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:30.269523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:30.269523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:30.269523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:30.269523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:30.269523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:30.269523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:30.269523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:30.269523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:30.269523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:30.269523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:30.269523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:30.269523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:30.270468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:30.270468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:30.270468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:30.270468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:30.270468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:30.270468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:30.270468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:30.270468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:30.270468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:30.270468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:30.270468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:30.270468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:26:30.271642 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 39/81] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:30.273451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4170861-4790-4595-b163-3df3764d14cf None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:30.278205 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab79c38-3875-4c80-a090-161ba069a460 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:30.313394 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01b6a4-ca76-4bf4-be46-ae5b7b3bdcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:30.314187 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01b6a4-ca76-4bf4-be46-ae5b7b3bdcb4 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:30.314479 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01b6a4-ca76-4bf4-be46-ae5b7b3bdcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:30.314753 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01b6a4-ca76-4bf4-be46-ae5b7b3bdcb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:30.315927 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01b6a4-ca76-4bf4-be46-ae5b7b3bdcb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:30.317388 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01b6a4-ca76-4bf4-be46-ae5b7b3bdcb4 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:30.319044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 43/82] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:30.319801 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01b6a4-ca76-4bf4-be46-ae5b7b3bdcb4 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:30.361687 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:30.378183 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4170861-4790-4595-b163-3df3764d14cf None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:30.411310 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1c0e7-b87d-4881-8908-dce7ec63b0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:30.411987 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1c0e7-b87d-4881-8908-dce7ec63b0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:30.412278 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1c0e7-b87d-4881-8908-dce7ec63b0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:30.412567 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1c0e7-b87d-4881-8908-dce7ec63b0f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:30.413339 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1c0e7-b87d-4881-8908-dce7ec63b0f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:30.413679 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1c0e7-b87d-4881-8908-dce7ec63b0f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:30.415062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1c0e7-b87d-4881-8908-dce7ec63b0f6 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:30.417601 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01b6a4-ca76-4bf4-be46-ae5b7b3bdcb4 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:30.424026 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 40/83] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:26:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:30.425471 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1c0e7-b87d-4881-8908-dce7ec63b0f6 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:30.435085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:30.453480 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41744a40-90ca-4ef9-9b81-a8b8cd409776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:30.454349 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41744a40-90ca-4ef9-9b81-a8b8cd409776 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:30.454636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41744a40-90ca-4ef9-9b81-a8b8cd409776 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:30.454915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41744a40-90ca-4ef9-9b81-a8b8cd409776 None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:30.456115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41744a40-90ca-4ef9-9b81-a8b8cd409776 None admin] RBAC: Authorizing `identity:get_project(project_id=24e47f2dd0c74c218f4fd2059483f785)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:30.457597 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41744a40-90ca-4ef9-9b81-a8b8cd409776 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:30.459279 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 44/84] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:30 2022] GET /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:30.460040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41744a40-90ca-4ef9-9b81-a8b8cd409776 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:30.525799 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1c0e7-b87d-4881-8908-dce7ec63b0f6 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:30.559409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d50bd-3c87-4a9f-8c44-14ba340f63ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:30.560282 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d50bd-3c87-4a9f-8c44-14ba340f63ec None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:30.560599 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d50bd-3c87-4a9f-8c44-14ba340f63ec None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:30.560888 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d50bd-3c87-4a9f-8c44-14ba340f63ec None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:30.561929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41744a40-90ca-4ef9-9b81-a8b8cd409776 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:30.562096 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d50bd-3c87-4a9f-8c44-14ba340f63ec None admin] RBAC: Authorizing `identity:get_project(project_id=24e47f2dd0c74c218f4fd2059483f785)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:30.563574 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d50bd-3c87-4a9f-8c44-14ba340f63ec None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:30.565339 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 41/85] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:30 2022] GET /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:30.566091 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d50bd-3c87-4a9f-8c44-14ba340f63ec None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:30.572932 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:30.596562 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc72b7-f335-4459-9268-994bb4c720d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:30.597265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc72b7-f335-4459-9268-994bb4c720d7 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:30.597559 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc72b7-f335-4459-9268-994bb4c720d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:30.597841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc72b7-f335-4459-9268-994bb4c720d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:30.598602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc72b7-f335-4459-9268-994bb4c720d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:30.598973 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc72b7-f335-4459-9268-994bb4c720d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:30.600469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc72b7-f335-4459-9268-994bb4c720d7 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:30.609453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 45/86] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:26:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:30.610249 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc72b7-f335-4459-9268-994bb4c720d7 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:30.644881 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:30.668490 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d50bd-3c87-4a9f-8c44-14ba340f63ec None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:30.702415 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44d3dd-a285-42f1-8f06-e8399d3d63b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:30.703160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44d3dd-a285-42f1-8f06-e8399d3d63b9 None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:30.703435 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44d3dd-a285-42f1-8f06-e8399d3d63b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:30.703715 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44d3dd-a285-42f1-8f06-e8399d3d63b9 None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:30.704439 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44d3dd-a285-42f1-8f06-e8399d3d63b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:30.705943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44d3dd-a285-42f1-8f06-e8399d3d63b9 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:30.711854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc72b7-f335-4459-9268-994bb4c720d7 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:30.744921 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ad05e-e716-419d-9201-e06c116bb3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:30.746440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ad05e-e716-419d-9201-e06c116bb3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:30.746734 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ad05e-e716-419d-9201-e06c116bb3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:30.747018 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ad05e-e716-419d-9201-e06c116bb3b0 None admin] PATH_INFO: `/v3/users/65e4eacde0544043820a4928b1ac4083` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:30.748223 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2ad05e-e716-419d-9201-e06c116bb3b0 None admin] RBAC: Authorizing `identity:get_user(user_id=65e4eacde0544043820a4928b1ac4083)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:30.749761 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2ad05e-e716-419d-9201-e06c116bb3b0 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:30.751398 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 46/87] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:30 2022] GET /identity/v3/users/65e4eacde0544043820a4928b1ac4083 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:30.752250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ad05e-e716-419d-9201-e06c116bb3b0 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:30.793090 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:30.799415 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 42/88] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 22:26:30 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:26:30.800403 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44d3dd-a285-42f1-8f06-e8399d3d63b9 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:30.859908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ad05e-e716-419d-9201-e06c116bb3b0 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:30.904288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8e57a-0845-48a1-bcfa-18cf6959a7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:30.905278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8e57a-0845-48a1-bcfa-18cf6959a7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:30.905576 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8e57a-0845-48a1-bcfa-18cf6959a7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:30.905861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8e57a-0845-48a1-bcfa-18cf6959a7c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:30.906871 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa8e57a-0845-48a1-bcfa-18cf6959a7c3 None admin] RBAC: Adding query filter params (role.id=6ff7708b59984b9287cc81d46a57b4b4, scope.project.id=24e47f2dd0c74c218f4fd2059483f785, user.id=091e3a1287aa48ddab1a4bfa5166599f) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:30.907487 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa8e57a-0845-48a1-bcfa-18cf6959a7c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:30.909015 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa8e57a-0845-48a1-bcfa-18cf6959a7c3 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:30.917759 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 47/89] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:30 2022] GET /identity/v3/role_assignments?user.id=091e3a1287aa48ddab1a4bfa5166599f&scope.project.id=24e47f2dd0c74c218f4fd2059483f785&role.id=6ff7708b59984b9287cc81d46a57b4b4 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:30.919092 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8e57a-0845-48a1-bcfa-18cf6959a7c3 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:30.928339 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44d3dd-a285-42f1-8f06-e8399d3d63b9 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:30.972048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:30.983701 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae5833-2ac4-4cd5-9af9-4a20b46960c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:30.983701 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae5833-2ac4-4cd5-9af9-4a20b46960c4 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:30.983984 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae5833-2ac4-4cd5-9af9-4a20b46960c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:30.984050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae5833-2ac4-4cd5-9af9-4a20b46960c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:30.985857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae5833-2ac4-4cd5-9af9-4a20b46960c4 None admin] RBAC: Adding query filter params (role.id=1305747c745d49dc9a9718b7aa8aa4a9, scope.project.id=24e47f2dd0c74c218f4fd2059483f785, user.id=65e4eacde0544043820a4928b1ac4083) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:30.985857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae5833-2ac4-4cd5-9af9-4a20b46960c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:30.990797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae5833-2ac4-4cd5-9af9-4a20b46960c4 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:30.993238 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 43/90] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:30 2022] GET /identity/v3/role_assignments?user.id=65e4eacde0544043820a4928b1ac4083&scope.project.id=24e47f2dd0c74c218f4fd2059483f785&role.id=1305747c745d49dc9a9718b7aa8aa4a9 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:30.993940 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae5833-2ac4-4cd5-9af9-4a20b46960c4 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:31.072252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8e57a-0845-48a1-bcfa-18cf6959a7c3 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:31.077528 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:31.127661 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b65f5-f7da-41d3-9ee2-8671df801ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:31.128569 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b65f5-f7da-41d3-9ee2-8671df801ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:31.128899 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b65f5-f7da-41d3-9ee2-8671df801ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:31.129185 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b65f5-f7da-41d3-9ee2-8671df801ef5 None admin] PATH_INFO: `/v3/users/65e4eacde0544043820a4928b1ac4083` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:31.132069 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b65f5-f7da-41d3-9ee2-8671df801ef5 None admin] RBAC: Authorizing `identity:get_user(user_id=65e4eacde0544043820a4928b1ac4083)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:31.133663 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b65f5-f7da-41d3-9ee2-8671df801ef5 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:31.135419 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 48/91] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:30 2022] GET /identity/v3/users/65e4eacde0544043820a4928b1ac4083 => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:31.139543 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae5833-2ac4-4cd5-9af9-4a20b46960c4 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:31.141192 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b65f5-f7da-41d3-9ee2-8671df801ef5 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:31.189864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b2f92-25ab-46f5-a19e-68884674f818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:31.191218 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b2f92-25ab-46f5-a19e-68884674f818 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:31.191702 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b2f92-25ab-46f5-a19e-68884674f818 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:31.192171 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b2f92-25ab-46f5-a19e-68884674f818 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:31.271476 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b65f5-f7da-41d3-9ee2-8671df801ef5 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:31.277028 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:31.295233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b2f92-25ab-46f5-a19e-68884674f818 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:31.296771 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b2f92-25ab-46f5-a19e-68884674f818 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:31.330576 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce77564-50ec-4486-804d-6e15fdfeb07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:31.332088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce77564-50ec-4486-804d-6e15fdfeb07a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:31.332534 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce77564-50ec-4486-804d-6e15fdfeb07a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:31.333002 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce77564-50ec-4486-804d-6e15fdfeb07a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:31.391872 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:31.431836 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce77564-50ec-4486-804d-6e15fdfeb07a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:31.436604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce77564-50ec-4486-804d-6e15fdfeb07a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:31.485292 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 44/92] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:26:31.493605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b2f92-25ab-46f5-a19e-68884674f818 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:31.540689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 49/93] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 22:26:31.547210 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce77564-50ec-4486-804d-6e15fdfeb07a None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:31.633081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b2f92-25ab-46f5-a19e-68884674f818 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:31.633424 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:31.635033 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:31.660209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce77564-50ec-4486-804d-6e15fdfeb07a None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:31.696560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4c0e5-c728-462d-98b1-edf85d4b837f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:31.697496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244c7d3-8415-4367-ac9c-3ec8d234a861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:31.697980 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4c0e5-c728-462d-98b1-edf85d4b837f None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:31.698936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4c0e5-c728-462d-98b1-edf85d4b837f None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:31.699166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244c7d3-8415-4367-ac9c-3ec8d234a861 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:31.699510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4c0e5-c728-462d-98b1-edf85d4b837f None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:31.699630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244c7d3-8415-4367-ac9c-3ec8d234a861 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:31.700006 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244c7d3-8415-4367-ac9c-3ec8d234a861 None admin] PATH_INFO: `/v3/projects/fa88275401a14e5aaa627843dc619cc7` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:31.705432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f4c0e5-c728-462d-98b1-edf85d4b837f None admin] RBAC: Authorizing `identity:get_project(project_id=24e47f2dd0c74c218f4fd2059483f785)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:31.707169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f4c0e5-c728-462d-98b1-edf85d4b837f None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:31.709397 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0244c7d3-8415-4367-ac9c-3ec8d234a861 None admin] RBAC: Authorizing `identity:get_project(project_id=fa88275401a14e5aaa627843dc619cc7)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:31.710878 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0244c7d3-8415-4367-ac9c-3ec8d234a861 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:31.711731 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 45/94] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:31 2022] GET /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:31.712993 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 50/95] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:31 2022] GET /identity/v3/projects/fa88275401a14e5aaa627843dc619cc7 => generated 312 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:31.718396 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4c0e5-c728-462d-98b1-edf85d4b837f None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:31.719572 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244c7d3-8415-4367-ac9c-3ec8d234a861 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:31.832082 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4c0e5-c728-462d-98b1-edf85d4b837f None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:31.885395 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5bd6c-f846-4c48-b363-d062445aef67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:31.886560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5bd6c-f846-4c48-b363-d062445aef67 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:31.886857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5bd6c-f846-4c48-b363-d062445aef67 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:31.887151 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5bd6c-f846-4c48-b363-d062445aef67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:31.888007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5bd6c-f846-4c48-b363-d062445aef67 None admin] RBAC: Adding query filter params (role.id=c3ad3cf6be504e35aaec10b7b9f3f34d, scope.project.id=24e47f2dd0c74c218f4fd2059483f785, user.id=65e4eacde0544043820a4928b1ac4083) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:31.888327 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5bd6c-f846-4c48-b363-d062445aef67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:31.889830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5bd6c-f846-4c48-b363-d062445aef67 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:31.900121 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 46/96] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:31 2022] GET /identity/v3/role_assignments?user.id=65e4eacde0544043820a4928b1ac4083&scope.project.id=24e47f2dd0c74c218f4fd2059483f785&role.id=c3ad3cf6be504e35aaec10b7b9f3f34d => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:32.000874 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244c7d3-8415-4367-ac9c-3ec8d234a861 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:32.040624 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb065521-f7e9-4edd-88e8-85710a08080d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:32.041334 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb065521-f7e9-4edd-88e8-85710a08080d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:32.041626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb065521-f7e9-4edd-88e8-85710a08080d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:32.042067 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb065521-f7e9-4edd-88e8-85710a08080d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:32.043399 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb065521-f7e9-4edd-88e8-85710a08080d None admin] RBAC: Adding query filter params (role.id=1305747c745d49dc9a9718b7aa8aa4a9, scope.project.id=fa88275401a14e5aaa627843dc619cc7, user.id=65e4eacde0544043820a4928b1ac4083) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:32.043717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb065521-f7e9-4edd-88e8-85710a08080d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:32.045468 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb065521-f7e9-4edd-88e8-85710a08080d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:32.051326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 51/97] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:31 2022] GET /identity/v3/role_assignments?user.id=65e4eacde0544043820a4928b1ac4083&scope.project.id=fa88275401a14e5aaa627843dc619cc7&role.id=1305747c745d49dc9a9718b7aa8aa4a9 => generated 259 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:33.777543 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fbb00-0207-49a6-bec2-0ada3aaf285b None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:33.808501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fbb00-0207-49a6-bec2-0ada3aaf285b None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:33.808501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fbb00-0207-49a6-bec2-0ada3aaf285b None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:33.808501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 47/98] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:33.808781 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b3bab-0efe-45f7-8ea6-6a6e69417b75 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:33.808781 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b3bab-0efe-45f7-8ea6-6a6e69417b75 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:33.808781 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b3bab-0efe-45f7-8ea6-6a6e69417b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:33.808781 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:33.809638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-d88b3bab-0efe-45f7-8ea6-6a6e69417b75 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:33.810880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d88b3bab-0efe-45f7-8ea6-6a6e69417b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:33.898988 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 52/99] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:33.920821 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84543ae-b8b7-4c5b-adb5-f0501bdd5ce2 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:33.921126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84543ae-b8b7-4c5b-adb5-f0501bdd5ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:33.921419 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84543ae-b8b7-4c5b-adb5-f0501bdd5ce2 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:33.923067 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 48/100] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:33.928344 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6901f-171e-41ef-814a-1ffb5fe5469e None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:33.928678 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6901f-171e-41ef-814a-1ffb5fe5469e None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:33.928969 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6901f-171e-41ef-814a-1ffb5fe5469e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:33.934739 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84543ae-b8b7-4c5b-adb5-f0501bdd5ce2 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:33.960483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-63e6901f-171e-41ef-814a-1ffb5fe5469e None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:33.961744 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-63e6901f-171e-41ef-814a-1ffb5fe5469e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:34.009011 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:34.136651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 53/101] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:34.138129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af78f5-5f71-4623-a551-417ef52e640f None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:34.138439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af78f5-5f71-4623-a551-417ef52e640f None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:34.138728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af78f5-5f71-4623-a551-417ef52e640f None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:34.140166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 54/102] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:34.145521 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c6027-7578-4742-9165-4edfad5b8547 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:34.145808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c6027-7578-4742-9165-4edfad5b8547 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:34.146093 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c6027-7578-4742-9165-4edfad5b8547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:34.172295 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-100c6027-7578-4742-9165-4edfad5b8547 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:34.173531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-100c6027-7578-4742-9165-4edfad5b8547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:34.190882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84543ae-b8b7-4c5b-adb5-f0501bdd5ce2 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:34.252790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a48001-75f1-4213-a893-4a860044cc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:34.255698 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a48001-75f1-4213-a893-4a860044cc62 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:34.255698 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a48001-75f1-4213-a893-4a860044cc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:34.255698 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a48001-75f1-4213-a893-4a860044cc62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:34.268720 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a48001-75f1-4213-a893-4a860044cc62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:34.274785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-07a48001-75f1-4213-a893-4a860044cc62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:34.274785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:34.274785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:34.274785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:34.274785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:34.274785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:34.274785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:34.274785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:34.274785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:34.274785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:34.274785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:34.274785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:34.274785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:34.274785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:34.274785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:34.274785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:34.275674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:34.275674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:34.275674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:34.275674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:34.275674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:34.275674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:34.275674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:34.275674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:34.275674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:34.275674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:34.275674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:34.275674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:34.275674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:34.275674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:34.275674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:34.275674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:34.276522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:34.276522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:34.276522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:34.276522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:34.276522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:34.276522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:34.276522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:34.276522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:34.276522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:34.276522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:34.276522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:34.276522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:26:34.276522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 49/103] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:34.276522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a48001-75f1-4213-a893-4a860044cc62 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:34.293944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:34.307136 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 55/104] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:26:34.320222 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c6027-7578-4742-9165-4edfad5b8547 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:34.528969 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:34.552818 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a48001-75f1-4213-a893-4a860044cc62 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:34.559103 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c6027-7578-4742-9165-4edfad5b8547 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:34.592767 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711068d5-1b1a-42c3-89ef-aea323c0e835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:34.593867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711068d5-1b1a-42c3-89ef-aea323c0e835 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:34.594167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711068d5-1b1a-42c3-89ef-aea323c0e835 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:34.594455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711068d5-1b1a-42c3-89ef-aea323c0e835 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:34.598298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947bc55-34f6-46d2-bb84-aad52163b171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:34.599233 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947bc55-34f6-46d2-bb84-aad52163b171 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:34.599732 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947bc55-34f6-46d2-bb84-aad52163b171 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:34.600219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947bc55-34f6-46d2-bb84-aad52163b171 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:34.601238 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947bc55-34f6-46d2-bb84-aad52163b171 None admin] RBAC: Adding query filter params (name=member) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:34.601786 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947bc55-34f6-46d2-bb84-aad52163b171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:34.603713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947bc55-34f6-46d2-bb84-aad52163b171 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:34.606890 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711068d5-1b1a-42c3-89ef-aea323c0e835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:34.612965 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 56/105] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:26:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:34.613627 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947bc55-34f6-46d2-bb84-aad52163b171 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:34.616736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-711068d5-1b1a-42c3-89ef-aea323c0e835 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:34.616736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:34.616736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:34.616736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:34.616736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:34.616736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:34.616736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:34.616736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:34.616736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:34.616736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:34.616736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:34.616736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:34.616736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:34.616736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:34.616736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:34.616736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:34.617631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:34.617631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:34.617631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:34.617631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:34.617631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:34.617631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:34.617631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:34.617631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:34.617631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:34.617631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:34.617631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:34.617631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:34.617631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:34.617631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:34.617631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:34.617631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:34.618406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:34.618406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:34.618406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:34.618406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:34.618406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:34.618406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:34.618406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:34.618406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:34.618406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:34.618406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:34.618406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:34.618406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:26:34.622103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 50/106] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:26:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:34.623580 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70914938-3c11-4de4-807f-ffbc98d2cda0 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:34.623878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70914938-3c11-4de4-807f-ffbc98d2cda0 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:34.624160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70914938-3c11-4de4-807f-ffbc98d2cda0 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:34.625634 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 51/107] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:34.626574 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef06b4-a6d2-4cac-b7e4-b420401fe96f None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:34.626864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef06b4-a6d2-4cac-b7e4-b420401fe96f None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:34.627149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef06b4-a6d2-4cac-b7e4-b420401fe96f None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:34.628511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 52/108] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:34.633358 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef06b4-a6d2-4cac-b7e4-b420401fe96f None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:34.686135 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:34.732772 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947bc55-34f6-46d2-bb84-aad52163b171 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:34.747459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef06b4-a6d2-4cac-b7e4-b420401fe96f None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:34.768275 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939c91b-1371-4b81-b569-8d3774bddd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:34.769113 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939c91b-1371-4b81-b569-8d3774bddd54 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:34.769418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939c91b-1371-4b81-b569-8d3774bddd54 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:34.769697 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939c91b-1371-4b81-b569-8d3774bddd54 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:34.780510 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939c91b-1371-4b81-b569-8d3774bddd54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:34.783927 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3784c32-caf2-4f02-97ef-2be0663772a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:34.784849 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3784c32-caf2-4f02-97ef-2be0663772a8 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:34.785140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3784c32-caf2-4f02-97ef-2be0663772a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:34.785424 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3784c32-caf2-4f02-97ef-2be0663772a8 None admin] PATH_INFO: `/v3/users/65e4eacde0544043820a4928b1ac4083` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:34.786626 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3784c32-caf2-4f02-97ef-2be0663772a8 None admin] RBAC: Authorizing `identity:get_user(user_id=65e4eacde0544043820a4928b1ac4083)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:34.788110 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3784c32-caf2-4f02-97ef-2be0663772a8 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:34.788351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-6939c91b-1371-4b81-b569-8d3774bddd54 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:26:34.788351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:34.788351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:34.788351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:34.788351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:34.788351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:34.788351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:34.788351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:34.788351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:34.788351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:34.788351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:34.788351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:34.788351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:34.788351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:34.788351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:34.788351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:34.789354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:34.789354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:34.789354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:34.789354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:34.789354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:34.789354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:34.789354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:34.789354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:34.789354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:34.789354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:34.789354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:34.789354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:34.789354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:34.789354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:34.789354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:34.789354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:34.790124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:34.790124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:34.790124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:34.790124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:34.790124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:34.790124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:34.790124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:34.790124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:34.790124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:34.790124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:34.790124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:26:34.790124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:26:34.790124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 57/109] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:34.791694 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 53/110] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:34 2022] GET /identity/v3/users/65e4eacde0544043820a4928b1ac4083 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:34.792486 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939c91b-1371-4b81-b569-8d3774bddd54 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:34.792685 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1a11c-00d7-4b11-be2e-1b1be6f8d58a None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:34.795697 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1a11c-00d7-4b11-be2e-1b1be6f8d58a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:34.796215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1a11c-00d7-4b11-be2e-1b1be6f8d58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:34.814810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-84f1a11c-00d7-4b11-be2e-1b1be6f8d58a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:34.816440 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-84f1a11c-00d7-4b11-be2e-1b1be6f8d58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:34.826891 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:34.886659 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 54/111] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:34.887759 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24608552-f2c0-4bdb-b418-f922377222a0 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:34.888058 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24608552-f2c0-4bdb-b418-f922377222a0 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:34.888344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24608552-f2c0-4bdb-b418-f922377222a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:34.904905 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-24608552-f2c0-4bdb-b418-f922377222a0 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:34.906205 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-24608552-f2c0-4bdb-b418-f922377222a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:34.918446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939c91b-1371-4b81-b569-8d3774bddd54 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:34.936233 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:34.959392 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c819c7-c0a5-495b-b421-6186fd644522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:34.960420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c819c7-c0a5-495b-b421-6186fd644522 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:34.960746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c819c7-c0a5-495b-b421-6186fd644522 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:34.961038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c819c7-c0a5-495b-b421-6186fd644522 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:34.961836 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c819c7-c0a5-495b-b421-6186fd644522 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:34.962153 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c819c7-c0a5-495b-b421-6186fd644522 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:34.964921 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c819c7-c0a5-495b-b421-6186fd644522 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:34.980604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 58/112] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:34.980604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c819c7-c0a5-495b-b421-6186fd644522 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:34.980859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 55/113] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:34.980859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24608552-f2c0-4bdb-b418-f922377222a0 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:35.060785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:35.086750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c819c7-c0a5-495b-b421-6186fd644522 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:35.087843 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24608552-f2c0-4bdb-b418-f922377222a0 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:35.122286 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b46ac-c79d-4048-8e40-12b9939182a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:35.122959 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b46ac-c79d-4048-8e40-12b9939182a8 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:35.123250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b46ac-c79d-4048-8e40-12b9939182a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:35.123535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b46ac-c79d-4048-8e40-12b9939182a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:35.123778 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14fb85-1bfc-4529-a8ca-1e5322ed1a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:35.124293 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b46ac-c79d-4048-8e40-12b9939182a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:35.124636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b46ac-c79d-4048-8e40-12b9939182a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:35.124742 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14fb85-1bfc-4529-a8ca-1e5322ed1a0f None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:35.124953 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14fb85-1bfc-4529-a8ca-1e5322ed1a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:35.125236 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14fb85-1bfc-4529-a8ca-1e5322ed1a0f None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:35.126011 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b46ac-c79d-4048-8e40-12b9939182a8 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:35.126424 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab14fb85-1bfc-4529-a8ca-1e5322ed1a0f None admin] RBAC: Authorizing `identity:get_project(project_id=24e47f2dd0c74c218f4fd2059483f785)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:35.127991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab14fb85-1bfc-4529-a8ca-1e5322ed1a0f None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:35.129723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 56/114] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:34 2022] GET /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:35.130491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14fb85-1bfc-4529-a8ca-1e5322ed1a0f None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:35.142224 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 59/115] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:26:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:35.142224 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b46ac-c79d-4048-8e40-12b9939182a8 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:35.178055 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:35.232933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14fb85-1bfc-4529-a8ca-1e5322ed1a0f None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:35.235547 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b46ac-c79d-4048-8e40-12b9939182a8 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:35.266262 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355adbd-7b2c-4eca-a00a-60ea2cb02f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:35.267064 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355adbd-7b2c-4eca-a00a-60ea2cb02f7a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:35.267365 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355adbd-7b2c-4eca-a00a-60ea2cb02f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:35.267649 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355adbd-7b2c-4eca-a00a-60ea2cb02f7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:35.268868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355adbd-7b2c-4eca-a00a-60ea2cb02f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:35.269048 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefccbb-91dd-4a4c-8128-be1850bbcead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:35.269887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefccbb-91dd-4a4c-8128-be1850bbcead None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:35.270176 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefccbb-91dd-4a4c-8128-be1850bbcead None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:35.270342 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355adbd-7b2c-4eca-a00a-60ea2cb02f7a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:35.270488 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefccbb-91dd-4a4c-8128-be1850bbcead None admin] PATH_INFO: `/v3/users/091e3a1287aa48ddab1a4bfa5166599f` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:35.271663 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefccbb-91dd-4a4c-8128-be1850bbcead None admin] RBAC: Authorizing `identity:get_user(user_id=091e3a1287aa48ddab1a4bfa5166599f)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:35.276756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 57/116] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:35.276756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355adbd-7b2c-4eca-a00a-60ea2cb02f7a None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:35.276991 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefccbb-91dd-4a4c-8128-be1850bbcead None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:35.276991 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 60/117] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:35 2022] GET /identity/v3/users/091e3a1287aa48ddab1a4bfa5166599f => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:35.276991 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d46c3-da2a-44c1-bdd1-8760d0663fa6 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:35.277176 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d46c3-da2a-44c1-bdd1-8760d0663fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:35.277439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d46c3-da2a-44c1-bdd1-8760d0663fa6 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:35.278755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 61/118] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:35.279566 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d46c3-da2a-44c1-bdd1-8760d0663fa6 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:35.289452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:35.377063 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355adbd-7b2c-4eca-a00a-60ea2cb02f7a None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:35.381185 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d46c3-da2a-44c1-bdd1-8760d0663fa6 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:35.390136 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:35.409748 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8f523-3095-48ce-90e6-e09bbe720a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:35.410544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8f523-3095-48ce-90e6-e09bbe720a85 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:35.410829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8f523-3095-48ce-90e6-e09bbe720a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:35.411107 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8f523-3095-48ce-90e6-e09bbe720a85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:35.413986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7c6eb-a951-440e-90a2-fbca0beecbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:35.414877 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7c6eb-a951-440e-90a2-fbca0beecbbb None admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:35.415168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7c6eb-a951-440e-90a2-fbca0beecbbb None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:35.415457 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7c6eb-a951-440e-90a2-fbca0beecbbb None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785/users/65e4eacde0544043820a4928b1ac4083/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:35.418167 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7c6eb-a951-440e-90a2-fbca0beecbbb None admin] RBAC: Authorizing `identity:create_grant(project_id=24e47f2dd0c74c218f4fd2059483f785, user_id=65e4eacde0544043820a4928b1ac4083, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:35.419955 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7c6eb-a951-440e-90a2-fbca0beecbbb None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:35.421809 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8f523-3095-48ce-90e6-e09bbe720a85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:35.429508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-77a8f523-3095-48ce-90e6-e09bbe720a85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:35.429508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:35.429508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:35.429508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:35.429508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:35.429508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:35.429508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:35.429508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:35.429508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:35.429508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:35.429508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:35.429508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:35.429508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:35.429508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:35.429508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:35.429508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:35.430508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:35.430508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:35.430508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:35.430508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:35.430508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:35.430508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:35.430508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:35.430508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:35.430508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:35.430508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:35.430508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:35.430508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:35.430508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:35.430508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:35.430508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:35.430508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:35.431562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:35.431562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:35.431562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:35.431562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:35.431562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:35.431562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:35.431562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:35.431562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:35.431562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:35.431562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:35.431562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:35.431562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:26:35.431562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 58/119] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:35.431562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8f523-3095-48ce-90e6-e09bbe720a85 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:35.489174 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 62/120] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:26:35 2022] PUT /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785/users/65e4eacde0544043820a4928b1ac4083/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:26:35.489985 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7c6eb-a951-440e-90a2-fbca0beecbbb None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:35.499994 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:35.535143 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8f523-3095-48ce-90e6-e09bbe720a85 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:35.573367 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea81de-5b77-4b92-b847-8e293617c18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:35.574313 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea81de-5b77-4b92-b847-8e293617c18b None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:35.574604 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea81de-5b77-4b92-b847-8e293617c18b None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:35.574890 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea81de-5b77-4b92-b847-8e293617c18b None admin] PATH_INFO: `/v3/users/be13617a9f5c4fd19704d8a065f5b59e` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:35.591202 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea81de-5b77-4b92-b847-8e293617c18b None admin] RBAC: Authorizing `identity:get_user(user_id=be13617a9f5c4fd19704d8a065f5b59e)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:35.592951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea81de-5b77-4b92-b847-8e293617c18b None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:35.594717 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 59/121] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:35 2022] GET /identity/v3/users/be13617a9f5c4fd19704d8a065f5b59e => generated 289 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:35.595535 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea81de-5b77-4b92-b847-8e293617c18b None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:35.610024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7c6eb-a951-440e-90a2-fbca0beecbbb None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:35.656907 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fe593-6f8a-4009-860e-7c322c7c2627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:35.657818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fe593-6f8a-4009-860e-7c322c7c2627 None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:35.658271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fe593-6f8a-4009-860e-7c322c7c2627 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:35.659148 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fe593-6f8a-4009-860e-7c322c7c2627 None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:35.660048 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447fe593-6f8a-4009-860e-7c322c7c2627 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:35.661536 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447fe593-6f8a-4009-860e-7c322c7c2627 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:35.707947 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea81de-5b77-4b92-b847-8e293617c18b None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:35.735920 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:35.742805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 63/122] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 22:26:35 2022] POST /identity/v3/projects => generated 302 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:26:35.744176 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa314f-444a-4291-8b22-03f04d644f02 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:35.744474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa314f-444a-4291-8b22-03f04d644f02 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:35.745087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa314f-444a-4291-8b22-03f04d644f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:35.753087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fc4f9-58e6-4153-ad8e-3e0a905ae46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:35.754719 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fc4f9-58e6-4153-ad8e-3e0a905ae46d None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:35.755308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fc4f9-58e6-4153-ad8e-3e0a905ae46d None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:35.756828 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fc4f9-58e6-4153-ad8e-3e0a905ae46d None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:35.758695 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fc4f9-58e6-4153-ad8e-3e0a905ae46d None admin] RBAC: Authorizing `identity:get_project(project_id=24e47f2dd0c74c218f4fd2059483f785)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:35.760465 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fc4f9-58e6-4153-ad8e-3e0a905ae46d None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:35.763460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 60/123] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:35 2022] GET /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:35.764561 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fc4f9-58e6-4153-ad8e-3e0a905ae46d None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:35.774081 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-f0fa314f-444a-4291-8b22-03f04d644f02 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:35.775935 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-f0fa314f-444a-4291-8b22-03f04d644f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:35.793097 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:35.875257 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 64/124] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:35.876569 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa314f-444a-4291-8b22-03f04d644f02 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:35.903459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fc4f9-58e6-4153-ad8e-3e0a905ae46d None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:35.963338 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f5f61-a307-475e-a714-9fdf46a30671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:35.965166 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f5f61-a307-475e-a714-9fdf46a30671 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:35.965664 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f5f61-a307-475e-a714-9fdf46a30671 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:35.966080 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f5f61-a307-475e-a714-9fdf46a30671 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:35.966798 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f5f61-a307-475e-a714-9fdf46a30671 None admin] RBAC: Adding query filter params (name=member) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:35.967121 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f5f61-a307-475e-a714-9fdf46a30671 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:35.968611 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f5f61-a307-475e-a714-9fdf46a30671 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:35.979005 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 61/125] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:26:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:35.980470 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f5f61-a307-475e-a714-9fdf46a30671 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:36.008275 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa314f-444a-4291-8b22-03f04d644f02 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:36.056226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cc76e-9f15-4f78-81c5-f5ac4e92934a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:36.056927 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cc76e-9f15-4f78-81c5-f5ac4e92934a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:36.057826 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cc76e-9f15-4f78-81c5-f5ac4e92934a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:36.058287 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cc76e-9f15-4f78-81c5-f5ac4e92934a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:36.076475 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:36.087435 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:36.127736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f5f61-a307-475e-a714-9fdf46a30671 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:36.155079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839cc76e-9f15-4f78-81c5-f5ac4e92934a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:36.157778 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839cc76e-9f15-4f78-81c5-f5ac4e92934a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:36.165570 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf6390-2b39-4fd7-bec3-9f6d7906b9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:36.166568 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf6390-2b39-4fd7-bec3-9f6d7906b9b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:36.167493 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf6390-2b39-4fd7-bec3-9f6d7906b9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:36.167856 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf6390-2b39-4fd7-bec3-9f6d7906b9b0 None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785/users/091e3a1287aa48ddab1a4bfa5166599f/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:36.170375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf6390-2b39-4fd7-bec3-9f6d7906b9b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=24e47f2dd0c74c218f4fd2059483f785, user_id=091e3a1287aa48ddab1a4bfa5166599f, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:36.171943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf6390-2b39-4fd7-bec3-9f6d7906b9b0 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:36.271714 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 65/126] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:26:36.271714 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cc76e-9f15-4f78-81c5-f5ac4e92934a None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:36.289266 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 62/127] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:26:35 2022] PUT /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785/users/091e3a1287aa48ddab1a4bfa5166599f/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:26:36.290517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf6390-2b39-4fd7-bec3-9f6d7906b9b0 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:36.383539 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:36.422888 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf6390-2b39-4fd7-bec3-9f6d7906b9b0 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:36.445754 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:36.467600 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d86a8-4296-497a-9ef4-7d7153adba72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:36.469666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d86a8-4296-497a-9ef4-7d7153adba72 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:36.470195 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d86a8-4296-497a-9ef4-7d7153adba72 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:36.470695 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d86a8-4296-497a-9ef4-7d7153adba72 None admin] PATH_INFO: `/v3/users/65e4eacde0544043820a4928b1ac4083` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:36.472377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d86a8-4296-497a-9ef4-7d7153adba72 None admin] RBAC: Authorizing `identity:get_user(user_id=65e4eacde0544043820a4928b1ac4083)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:36.474284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d86a8-4296-497a-9ef4-7d7153adba72 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:36.480977 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 63/128] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:36 2022] GET /identity/v3/users/65e4eacde0544043820a4928b1ac4083 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:36.482446 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d86a8-4296-497a-9ef4-7d7153adba72 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:36.515742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cc76e-9f15-4f78-81c5-f5ac4e92934a None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:36.580320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fecf9-0e4c-48fc-943e-8a2a24f1c235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:36.581203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fecf9-0e4c-48fc-943e-8a2a24f1c235 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:36.581496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fecf9-0e4c-48fc-943e-8a2a24f1c235 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:36.581775 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fecf9-0e4c-48fc-943e-8a2a24f1c235 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:36.592871 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6fecf9-0e4c-48fc-943e-8a2a24f1c235 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:36.603756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-1a6fecf9-0e4c-48fc-943e-8a2a24f1c235 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:26:36.603756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:36.603756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:36.603756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:36.603756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:36.603756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:36.603756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:36.603756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:36.603756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:36.603756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:36.603756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:36.603756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:36.603756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:36.603756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:36.603756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:36.603756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:36.605276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:36.605276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:36.605276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:36.605276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:36.605276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:36.605276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:36.605276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:36.605276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:36.605276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:36.605276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:36.605276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:36.605276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:36.605276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:36.605276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:36.605276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:36.605276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:36.606057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:36.606057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:36.606057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:36.606057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:36.606057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:36.606057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:36.606057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:36.606057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:36.606057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:36.606057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:36.606057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:26:36.606057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:26:36.606057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 66/129] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:26:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:36.606682 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fecf9-0e4c-48fc-943e-8a2a24f1c235 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:36.608764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d86a8-4296-497a-9ef4-7d7153adba72 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:36.654822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078252f-9b8b-477f-a51d-d20d68b83b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:36.655721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078252f-9b8b-477f-a51d-d20d68b83b66 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:36.656015 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078252f-9b8b-477f-a51d-d20d68b83b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:36.656300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078252f-9b8b-477f-a51d-d20d68b83b66 None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:36.657585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078252f-9b8b-477f-a51d-d20d68b83b66 None admin] RBAC: Authorizing `identity:get_project(project_id=24e47f2dd0c74c218f4fd2059483f785)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:36.660209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078252f-9b8b-477f-a51d-d20d68b83b66 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:36.662435 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 64/130] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:36 2022] GET /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:36.663254 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078252f-9b8b-477f-a51d-d20d68b83b66 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:36.772124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:36.777167 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fecf9-0e4c-48fc-943e-8a2a24f1c235 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:36.792469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:36.838560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8463c-f0f0-40ec-ac46-32cb14be3b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:36.839733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8463c-f0f0-40ec-ac46-32cb14be3b7c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:36.840026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8463c-f0f0-40ec-ac46-32cb14be3b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:36.841040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8463c-f0f0-40ec-ac46-32cb14be3b7c None admin] PATH_INFO: `/v3/projects/fa88275401a14e5aaa627843dc619cc7` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:36.843584 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a8463c-f0f0-40ec-ac46-32cb14be3b7c None admin] RBAC: Authorizing `identity:get_project(project_id=fa88275401a14e5aaa627843dc619cc7)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:36.846761 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a8463c-f0f0-40ec-ac46-32cb14be3b7c None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:36.855766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 67/131] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:36 2022] GET /identity/v3/projects/fa88275401a14e5aaa627843dc619cc7 => generated 312 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:36.856696 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8463c-f0f0-40ec-ac46-32cb14be3b7c None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:36.861908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078252f-9b8b-477f-a51d-d20d68b83b66 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:36.968844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d308c-cd66-47f9-9d2a-98e0b3918827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:36.969593 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d308c-cd66-47f9-9d2a-98e0b3918827 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:36.969817 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d308c-cd66-47f9-9d2a-98e0b3918827 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:36.970109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d308c-cd66-47f9-9d2a-98e0b3918827 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:36.970877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d308c-cd66-47f9-9d2a-98e0b3918827 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:36.971198 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d308c-cd66-47f9-9d2a-98e0b3918827 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:36.972625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d308c-cd66-47f9-9d2a-98e0b3918827 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:36.980741 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8463c-f0f0-40ec-ac46-32cb14be3b7c None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:36.995397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 65/132] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:26:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:36.996216 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d308c-cd66-47f9-9d2a-98e0b3918827 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:37.037952 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0b118-28de-48b7-987e-0ab113a612f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:37.040419 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0b118-28de-48b7-987e-0ab113a612f3 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:37.040941 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0b118-28de-48b7-987e-0ab113a612f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:37.042190 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0b118-28de-48b7-987e-0ab113a612f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:37.044741 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0b118-28de-48b7-987e-0ab113a612f3 None admin] RBAC: Adding query filter params (role.id=917b4049bffd4db282dfe80ac37538be, scope.project.id=24e47f2dd0c74c218f4fd2059483f785, user.id=be13617a9f5c4fd19704d8a065f5b59e) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:37.047039 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0b118-28de-48b7-987e-0ab113a612f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:37.048773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0b118-28de-48b7-987e-0ab113a612f3 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:37.071535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 68/133] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:36 2022] GET /identity/v3/role_assignments?user.id=be13617a9f5c4fd19704d8a065f5b59e&scope.project.id=24e47f2dd0c74c218f4fd2059483f785&role.id=917b4049bffd4db282dfe80ac37538be => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:37.072798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0b118-28de-48b7-987e-0ab113a612f3 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:37.168100 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:37.200232 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d308c-cd66-47f9-9d2a-98e0b3918827 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:37.228991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:37.240571 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a15eb-7d58-46a4-92d1-a4156001fb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:37.241938 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a15eb-7d58-46a4-92d1-a4156001fb3a None admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:37.242470 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a15eb-7d58-46a4-92d1-a4156001fb3a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:37.242949 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a15eb-7d58-46a4-92d1-a4156001fb3a None admin] PATH_INFO: `/v3/projects/fa88275401a14e5aaa627843dc619cc7/users/65e4eacde0544043820a4928b1ac4083/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:37.245793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a15eb-7d58-46a4-92d1-a4156001fb3a None admin] RBAC: Authorizing `identity:create_grant(project_id=fa88275401a14e5aaa627843dc619cc7, user_id=65e4eacde0544043820a4928b1ac4083, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:37.247576 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a15eb-7d58-46a4-92d1-a4156001fb3a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:37.269657 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0b118-28de-48b7-987e-0ab113a612f3 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:37.321761 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3168108-aaeb-4952-a81c-8aebc52a2da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:37.323149 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3168108-aaeb-4952-a81c-8aebc52a2da0 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:37.323634 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3168108-aaeb-4952-a81c-8aebc52a2da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:37.324265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3168108-aaeb-4952-a81c-8aebc52a2da0 None admin] PATH_INFO: `/v3/users/65e4eacde0544043820a4928b1ac4083` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:37.325757 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3168108-aaeb-4952-a81c-8aebc52a2da0 None admin] RBAC: Authorizing `identity:get_user(user_id=65e4eacde0544043820a4928b1ac4083)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:37.327453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3168108-aaeb-4952-a81c-8aebc52a2da0 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:37.336279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 66/134] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:26:36 2022] PUT /identity/v3/projects/fa88275401a14e5aaa627843dc619cc7/users/65e4eacde0544043820a4928b1ac4083/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:26:37.338156 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 69/135] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:37 2022] GET /identity/v3/users/65e4eacde0544043820a4928b1ac4083 => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:37.344969 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a15eb-7d58-46a4-92d1-a4156001fb3a None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:37.481699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a15eb-7d58-46a4-92d1-a4156001fb3a None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:37.521211 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848af2d-8c06-4db6-8c9d-e022e227abfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:37.522123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848af2d-8c06-4db6-8c9d-e022e227abfd None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:37.522413 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848af2d-8c06-4db6-8c9d-e022e227abfd None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:37.522701 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848af2d-8c06-4db6-8c9d-e022e227abfd None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:37.524004 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848af2d-8c06-4db6-8c9d-e022e227abfd None admin] RBAC: Authorizing `identity:get_project(project_id=24e47f2dd0c74c218f4fd2059483f785)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:37.525528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848af2d-8c06-4db6-8c9d-e022e227abfd None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:37.528390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 67/136] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:37 2022] GET /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:37.534308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3168108-aaeb-4952-a81c-8aebc52a2da0 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:37.646744 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3168108-aaeb-4952-a81c-8aebc52a2da0 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:37.658332 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:37.684648 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced47725-25bd-4c3e-a48e-3fb8c1d6d3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:37.686009 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced47725-25bd-4c3e-a48e-3fb8c1d6d3b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:37.686411 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced47725-25bd-4c3e-a48e-3fb8c1d6d3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:37.686800 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced47725-25bd-4c3e-a48e-3fb8c1d6d3b2 None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785/users/65e4eacde0544043820a4928b1ac4083/roles/c3ad3cf6be504e35aaec10b7b9f3f34d` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:37.689756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced47725-25bd-4c3e-a48e-3fb8c1d6d3b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=24e47f2dd0c74c218f4fd2059483f785, user_id=65e4eacde0544043820a4928b1ac4083, role_id=c3ad3cf6be504e35aaec10b7b9f3f34d)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:37.691224 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced47725-25bd-4c3e-a48e-3fb8c1d6d3b2 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:37.757166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 70/137] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:26:37 2022] PUT /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785/users/65e4eacde0544043820a4928b1ac4083/roles/c3ad3cf6be504e35aaec10b7b9f3f34d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:26:37.828159 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187de312-1bdc-4b79-b587-ffa5e585864e None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:37.828489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187de312-1bdc-4b79-b587-ffa5e585864e None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:37.828800 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187de312-1bdc-4b79-b587-ffa5e585864e None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:37.830328 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 68/138] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:37.846720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb0728-879a-4d3d-8142-ddc078b4b842 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:37.847027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb0728-879a-4d3d-8142-ddc078b4b842 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:37.847314 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb0728-879a-4d3d-8142-ddc078b4b842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:37.872040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-cbbb0728-879a-4d3d-8142-ddc078b4b842 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:37.876168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-cbbb0728-879a-4d3d-8142-ddc078b4b842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:37.975558 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 71/139] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:38.012028 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187de312-1bdc-4b79-b587-ffa5e585864e None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:38.074201 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:38.146011 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187de312-1bdc-4b79-b587-ffa5e585864e None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:38.196070 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd1eac-f33c-4528-b5ab-644d0cb07ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:38.196920 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd1eac-f33c-4528-b5ab-644d0cb07ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:38.197218 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd1eac-f33c-4528-b5ab-644d0cb07ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:38.197507 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd1eac-f33c-4528-b5ab-644d0cb07ae7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:38.210318 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd1eac-f33c-4528-b5ab-644d0cb07ae7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:38.212029 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e2daf-d6d4-4249-9e94-391357d1f881 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:38.212462 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e2daf-d6d4-4249-9e94-391357d1f881 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:38.212659 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e2daf-d6d4-4249-9e94-391357d1f881 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:38.214881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 72/140] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:38.219885 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241cdec-93eb-4c8a-bc65-12c0f49f92de None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:38.220259 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241cdec-93eb-4c8a-bc65-12c0f49f92de None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:38.220560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241cdec-93eb-4c8a-bc65-12c0f49f92de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:38.225127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-a9dd1eac-f33c-4528-b5ab-644d0cb07ae7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:38.225127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:38.225127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:38.225127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:38.225127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:38.225127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:38.225127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:38.225127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:38.225127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:38.225127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:38.225127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:38.225127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:38.225127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:38.225127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:38.225127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:38.225127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:38.225972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:38.225972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:38.225972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:38.225972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:38.225972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:38.225972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:38.225972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:38.225972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:38.225972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:38.225972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:38.225972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:38.225972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:38.225972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:38.225972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:38.225972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:38.225972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:38.226791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:38.226791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:38.226791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:38.226791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:38.226791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:38.226791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:38.226791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:38.226791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:38.226791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:38.226791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:38.226791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:38.226791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:26:38.229864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 69/141] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:38.235924 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd1eac-f33c-4528-b5ab-644d0cb07ae7 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:38.252609 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-e241cdec-93eb-4c8a-bc65-12c0f49f92de None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:38.253807 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e241cdec-93eb-4c8a-bc65-12c0f49f92de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:38.369360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 73/142] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:26:38.433403 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241cdec-93eb-4c8a-bc65-12c0f49f92de None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:38.455072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd1eac-f33c-4528-b5ab-644d0cb07ae7 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:38.461783 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:38.491336 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:38.542547 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ed558-76e2-4283-b086-11850846d1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:38.543265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ed558-76e2-4283-b086-11850846d1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:38.543556 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ed558-76e2-4283-b086-11850846d1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:38.543836 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ed558-76e2-4283-b086-11850846d1c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:38.544896 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ed558-76e2-4283-b086-11850846d1c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:38.545210 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ed558-76e2-4283-b086-11850846d1c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:38.546622 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ed558-76e2-4283-b086-11850846d1c3 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:38.574609 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 70/143] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:26:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:38.580402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ed558-76e2-4283-b086-11850846d1c3 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:38.602582 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241cdec-93eb-4c8a-bc65-12c0f49f92de None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:38.661181 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7efc7b8-0de1-4ad3-9037-caf5ac2d9a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:38.661999 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efc7b8-0de1-4ad3-9037-caf5ac2d9a50 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:38.662287 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efc7b8-0de1-4ad3-9037-caf5ac2d9a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:38.662565 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efc7b8-0de1-4ad3-9037-caf5ac2d9a50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:38.663773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7efc7b8-0de1-4ad3-9037-caf5ac2d9a50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:38.665784 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7efc7b8-0de1-4ad3-9037-caf5ac2d9a50 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:38.668868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 74/144] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:38.674199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7efc7b8-0de1-4ad3-9037-caf5ac2d9a50 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:38.803880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7efc7b8-0de1-4ad3-9037-caf5ac2d9a50 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:38.805778 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:38.882818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f08d3-bade-4823-9c91-8495aef106de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:38.883917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f08d3-bade-4823-9c91-8495aef106de None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:38.884439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f08d3-bade-4823-9c91-8495aef106de None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:38.884982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f08d3-bade-4823-9c91-8495aef106de None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:38.886512 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f08d3-bade-4823-9c91-8495aef106de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:38.889723 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f08d3-bade-4823-9c91-8495aef106de None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:38.905915 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ed558-76e2-4283-b086-11850846d1c3 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:38.948083 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0548b-93a5-40f2-9b16-2e2fee0ea376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:38.949499 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0548b-93a5-40f2-9b16-2e2fee0ea376 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:38.949983 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0548b-93a5-40f2-9b16-2e2fee0ea376 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:38.950428 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0548b-93a5-40f2-9b16-2e2fee0ea376 None admin] PATH_INFO: `/v3/users/65e4eacde0544043820a4928b1ac4083` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:38.951910 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0548b-93a5-40f2-9b16-2e2fee0ea376 None admin] RBAC: Authorizing `identity:get_user(user_id=65e4eacde0544043820a4928b1ac4083)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:38.953575 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0548b-93a5-40f2-9b16-2e2fee0ea376 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:38.955438 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 71/145] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:38 2022] GET /identity/v3/users/65e4eacde0544043820a4928b1ac4083 => generated 275 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:38.960925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0548b-93a5-40f2-9b16-2e2fee0ea376 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:39.008273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 75/146] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 22:26:38 2022] POST /identity/v3/users => generated 283 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:26:39.021183 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:39.115171 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0548b-93a5-40f2-9b16-2e2fee0ea376 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:39.158036 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c71ec7-de6a-4553-a900-7b066124b34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:39.158763 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c71ec7-de6a-4553-a900-7b066124b34a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:39.159063 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c71ec7-de6a-4553-a900-7b066124b34a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:39.159352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c71ec7-de6a-4553-a900-7b066124b34a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:39.187615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6bf22-2048-4b44-9a1c-488b9694390d None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:39.187916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6bf22-2048-4b44-9a1c-488b9694390d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:39.188192 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6bf22-2048-4b44-9a1c-488b9694390d None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:39.189653 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 76/147] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:26:39.195104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ba51d-0a7a-45a8-9ea7-8f6a2a564f43 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:39.195398 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ba51d-0a7a-45a8-9ea7-8f6a2a564f43 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:39.195686 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ba51d-0a7a-45a8-9ea7-8f6a2a564f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:39.218164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-b52ba51d-0a7a-45a8-9ea7-8f6a2a564f43 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:39.221143 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-b52ba51d-0a7a-45a8-9ea7-8f6a2a564f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:39.295079 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c71ec7-de6a-4553-a900-7b066124b34a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:39.296608 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c71ec7-de6a-4553-a900-7b066124b34a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:39.327521 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 77/148] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:26:39.352485 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ba51d-0a7a-45a8-9ea7-8f6a2a564f43 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:39.353257 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:39.445012 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:39.460282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ba51d-0a7a-45a8-9ea7-8f6a2a564f43 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:39.487568 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 72/149] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 22:26:39.493462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c71ec7-de6a-4553-a900-7b066124b34a None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:39.497959 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b305be4-b65a-46c4-974f-ad025e253a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:39.498840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b305be4-b65a-46c4-974f-ad025e253a00 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:39.499130 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b305be4-b65a-46c4-974f-ad025e253a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:39.499417 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b305be4-b65a-46c4-974f-ad025e253a00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:39.514139 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b305be4-b65a-46c4-974f-ad025e253a00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:39.522360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-6b305be4-b65a-46c4-974f-ad025e253a00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:39.522360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:39.522360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:39.522360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:39.522360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:39.522360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:39.522360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:39.522360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:39.522360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:39.522360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:39.522360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:39.522360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:39.522360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:39.522360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:39.522360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:39.522360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:39.523209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:39.523209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:39.523209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:39.523209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:39.523209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:39.523209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:39.523209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:39.523209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:39.523209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:39.523209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:39.523209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:39.523209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:39.523209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:39.523209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:39.523209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:39.523209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:39.524189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:39.524189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:39.524189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:39.524189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:39.524189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:39.524189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:39.524189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:39.524189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:39.524189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:39.524189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:39.524189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:39.524189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:26:39.524189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 78/150] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:26:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:39.530743 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b305be4-b65a-46c4-974f-ad025e253a00 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:39.646735 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b305be4-b65a-46c4-974f-ad025e253a00 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:39.689444 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d0c45-7f49-4c38-bae0-1ed9c319535b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:39.690143 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d0c45-7f49-4c38-bae0-1ed9c319535b None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:39.690441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d0c45-7f49-4c38-bae0-1ed9c319535b None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:39.690788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d0c45-7f49-4c38-bae0-1ed9c319535b None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:39.691759 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46d0c45-7f49-4c38-bae0-1ed9c319535b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:39.692095 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46d0c45-7f49-4c38-bae0-1ed9c319535b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:39.693543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46d0c45-7f49-4c38-bae0-1ed9c319535b None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:39.703227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 79/151] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:39.708803 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d0c45-7f49-4c38-bae0-1ed9c319535b None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:39.727379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:39.825202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d0c45-7f49-4c38-bae0-1ed9c319535b None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:39.853848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c71ec7-de6a-4553-a900-7b066124b34a None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:39.864205 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27769f3f-9f55-44d7-a89b-3c978124b672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:39.865121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27769f3f-9f55-44d7-a89b-3c978124b672 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:39.865409 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27769f3f-9f55-44d7-a89b-3c978124b672 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:39.865693 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27769f3f-9f55-44d7-a89b-3c978124b672 None admin] PATH_INFO: `/v3/users/091e3a1287aa48ddab1a4bfa5166599f` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:39.866989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27769f3f-9f55-44d7-a89b-3c978124b672 None admin] RBAC: Authorizing `identity:get_user(user_id=091e3a1287aa48ddab1a4bfa5166599f)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:39.868529 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27769f3f-9f55-44d7-a89b-3c978124b672 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:39.871096 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 80/152] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:39 2022] GET /identity/v3/users/091e3a1287aa48ddab1a4bfa5166599f => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:39.877468 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27769f3f-9f55-44d7-a89b-3c978124b672 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:39.961375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569c826-99dd-478c-bd36-306a37b16096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:39.962191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569c826-99dd-478c-bd36-306a37b16096 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:39.962474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569c826-99dd-478c-bd36-306a37b16096 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:39.962748 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569c826-99dd-478c-bd36-306a37b16096 None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:39.964113 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569c826-99dd-478c-bd36-306a37b16096 None admin] RBAC: Authorizing `identity:get_project(project_id=24e47f2dd0c74c218f4fd2059483f785)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:39.965617 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569c826-99dd-478c-bd36-306a37b16096 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:39.967650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 73/153] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:39 2022] GET /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785 => generated 298 bytes in 474 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:39.967797 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:39.975756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569c826-99dd-478c-bd36-306a37b16096 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:39.983460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:39.993301 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27769f3f-9f55-44d7-a89b-3c978124b672 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:40.034016 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d648848-7c08-4f27-96dd-2b7317d96abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:40.035184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d648848-7c08-4f27-96dd-2b7317d96abc None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:40.035677 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d648848-7c08-4f27-96dd-2b7317d96abc None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:40.036152 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d648848-7c08-4f27-96dd-2b7317d96abc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:40.114396 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d648848-7c08-4f27-96dd-2b7317d96abc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:40.117035 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d648848-7c08-4f27-96dd-2b7317d96abc None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:40.183398 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569c826-99dd-478c-bd36-306a37b16096 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:40.201012 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:40.229801 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 81/154] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 22:26:40.231873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93033f5-d9b9-498b-ae44-55634fb84b7a None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:40.232370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93033f5-d9b9-498b-ae44-55634fb84b7a None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:40.237129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93033f5-d9b9-498b-ae44-55634fb84b7a None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:40.238751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 82/155] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:40.240291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93033f5-d9b9-498b-ae44-55634fb84b7a None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:40.293322 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83798c-078d-45cd-920d-9a12342c7b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:40.294009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83798c-078d-45cd-920d-9a12342c7b23 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:40.294555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83798c-078d-45cd-920d-9a12342c7b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:40.294842 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83798c-078d-45cd-920d-9a12342c7b23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:40.295785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83798c-078d-45cd-920d-9a12342c7b23 None admin] RBAC: Adding query filter params (role.id=1305747c745d49dc9a9718b7aa8aa4a9, scope.project.id=24e47f2dd0c74c218f4fd2059483f785, user.id=65e4eacde0544043820a4928b1ac4083) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:40.296017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83798c-078d-45cd-920d-9a12342c7b23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:40.297496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83798c-078d-45cd-920d-9a12342c7b23 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:40.308855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 74/156] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:39 2022] GET /identity/v3/role_assignments?user.id=65e4eacde0544043820a4928b1ac4083&scope.project.id=24e47f2dd0c74c218f4fd2059483f785&role.id=1305747c745d49dc9a9718b7aa8aa4a9 => generated 608 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:40.310431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069a1ac-6fcb-401a-b2ef-82c8fc84aeaf None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:40.310690 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069a1ac-6fcb-401a-b2ef-82c8fc84aeaf None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:40.310977 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069a1ac-6fcb-401a-b2ef-82c8fc84aeaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:40.341719 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-3069a1ac-6fcb-401a-b2ef-82c8fc84aeaf None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:40.342913 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3069a1ac-6fcb-401a-b2ef-82c8fc84aeaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:40.362117 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93033f5-d9b9-498b-ae44-55634fb84b7a None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:40.404688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d3407-08da-48b2-b17d-c58ac4a74f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:40.406010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d3407-08da-48b2-b17d-c58ac4a74f7c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:40.406604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d3407-08da-48b2-b17d-c58ac4a74f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:40.407078 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d3407-08da-48b2-b17d-c58ac4a74f7c None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:40.408623 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d3407-08da-48b2-b17d-c58ac4a74f7c None admin] RBAC: Authorizing `identity:get_project(project_id=24e47f2dd0c74c218f4fd2059483f785)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:40.410324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d3407-08da-48b2-b17d-c58ac4a74f7c None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:40.412416 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 83/157] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:40 2022] GET /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:40.418876 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d3407-08da-48b2-b17d-c58ac4a74f7c None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:40.443053 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:40.455963 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 75/158] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:40.485021 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:40.498714 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069a1ac-6fcb-401a-b2ef-82c8fc84aeaf None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:40.536984 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d3407-08da-48b2-b17d-c58ac4a74f7c None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:40.582022 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384df8b-0870-43ec-bc11-0cad6e8f4fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:40.582724 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384df8b-0870-43ec-bc11-0cad6e8f4fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:40.583019 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384df8b-0870-43ec-bc11-0cad6e8f4fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:40.583703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384df8b-0870-43ec-bc11-0cad6e8f4fc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:40.584711 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384df8b-0870-43ec-bc11-0cad6e8f4fc4 None admin] RBAC: Adding query filter params (role.id=6ff7708b59984b9287cc81d46a57b4b4, scope.project.id=24e47f2dd0c74c218f4fd2059483f785, user.id=091e3a1287aa48ddab1a4bfa5166599f) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:40.585213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384df8b-0870-43ec-bc11-0cad6e8f4fc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:40.586837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384df8b-0870-43ec-bc11-0cad6e8f4fc4 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:40.594479 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 84/159] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:40 2022] GET /identity/v3/role_assignments?user.id=091e3a1287aa48ddab1a4bfa5166599f&scope.project.id=24e47f2dd0c74c218f4fd2059483f785&role.id=6ff7708b59984b9287cc81d46a57b4b4 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:40.670163 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069a1ac-6fcb-401a-b2ef-82c8fc84aeaf None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:40.713007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e949b45-03d5-48c2-ad42-fe6789db37f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:40.713883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e949b45-03d5-48c2-ad42-fe6789db37f0 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:40.714171 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e949b45-03d5-48c2-ad42-fe6789db37f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:40.714454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e949b45-03d5-48c2-ad42-fe6789db37f0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:40.726909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e949b45-03d5-48c2-ad42-fe6789db37f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:40.735699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-9e949b45-03d5-48c2-ad42-fe6789db37f0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:26:40.735699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:40.735699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:40.735699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:40.735699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:40.735699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:40.735699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:40.735699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:40.735699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:40.735699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:40.735699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:40.735699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:40.735699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:40.735699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:40.735699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:40.735699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:40.736692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:40.736692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:40.736692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:40.736692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:40.736692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:40.736692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:40.736692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:40.736692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:40.736692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:40.736692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:40.736692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:40.736692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:40.736692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:40.736692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:40.736692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:40.736692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:40.737565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:40.737565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:40.737565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:40.737565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:40.737565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:40.737565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:40.737565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:40.737565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:40.737565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:40.737565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:40.737565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:26:40.737565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:26:40.738995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 76/160] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:26:40.745404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384df8b-0870-43ec-bc11-0cad6e8f4fc4 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:40.859610 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384df8b-0870-43ec-bc11-0cad6e8f4fc4 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:40.867284 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:40.895680 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ae9bf-8d70-4991-8c7a-77c481b11ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:40.896364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ae9bf-8d70-4991-8c7a-77c481b11ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:40.896715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ae9bf-8d70-4991-8c7a-77c481b11ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:40.896994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ae9bf-8d70-4991-8c7a-77c481b11ee2 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:40.897752 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ae9bf-8d70-4991-8c7a-77c481b11ee2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:40.898114 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ae9bf-8d70-4991-8c7a-77c481b11ee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:40.899584 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ae9bf-8d70-4991-8c7a-77c481b11ee2 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:40.910473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 85/161] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:26:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:40.915610 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e949b45-03d5-48c2-ad42-fe6789db37f0 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:40.979682 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:41.032197 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e949b45-03d5-48c2-ad42-fe6789db37f0 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:41.078564 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd47bd-5d14-4178-9bbc-755d4b4372e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:41.079450 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd47bd-5d14-4178-9bbc-755d4b4372e6 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:41.079732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd47bd-5d14-4178-9bbc-755d4b4372e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:41.080007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd47bd-5d14-4178-9bbc-755d4b4372e6 None admin] PATH_INFO: `/v3/users/be13617a9f5c4fd19704d8a065f5b59e` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:41.081255 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd47bd-5d14-4178-9bbc-755d4b4372e6 None admin] RBAC: Authorizing `identity:get_user(user_id=be13617a9f5c4fd19704d8a065f5b59e)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:41.082711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd47bd-5d14-4178-9bbc-755d4b4372e6 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:41.084348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 77/162] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:40 2022] GET /identity/v3/users/be13617a9f5c4fd19704d8a065f5b59e => generated 289 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:41.090131 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ae9bf-8d70-4991-8c7a-77c481b11ee2 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:41.205601 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ae9bf-8d70-4991-8c7a-77c481b11ee2 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:41.240931 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e1ade-4833-4825-be2b-e60a37530acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:41.241803 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e1ade-4833-4825-be2b-e60a37530acc None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:41.242082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e1ade-4833-4825-be2b-e60a37530acc None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:41.242356 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e1ade-4833-4825-be2b-e60a37530acc None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:41.243558 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e1ade-4833-4825-be2b-e60a37530acc None admin] RBAC: Authorizing `identity:get_project(project_id=24e47f2dd0c74c218f4fd2059483f785)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:41.245589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e1ade-4833-4825-be2b-e60a37530acc None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:41.247296 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 86/163] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:41 2022] GET /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:41.253009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd47bd-5d14-4178-9bbc-755d4b4372e6 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:41.378836 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd47bd-5d14-4178-9bbc-755d4b4372e6 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:41.415023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:41.428869 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e89ec-65ee-44eb-9acc-87125bbbd7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:41.429786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e89ec-65ee-44eb-9acc-87125bbbd7c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:41.430077 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e89ec-65ee-44eb-9acc-87125bbbd7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:41.430361 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e89ec-65ee-44eb-9acc-87125bbbd7c2 None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785/users/be13617a9f5c4fd19704d8a065f5b59e/roles/917b4049bffd4db282dfe80ac37538be` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:41.437970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e89ec-65ee-44eb-9acc-87125bbbd7c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=24e47f2dd0c74c218f4fd2059483f785, user_id=be13617a9f5c4fd19704d8a065f5b59e, role_id=917b4049bffd4db282dfe80ac37538be)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:41.439452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e89ec-65ee-44eb-9acc-87125bbbd7c2 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:41.478006 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67ff8f-6fce-401f-a19e-2718dd721034 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:41.478317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67ff8f-6fce-401f-a19e-2718dd721034 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:41.478569 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67ff8f-6fce-401f-a19e-2718dd721034 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:41.480975 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 87/164] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:41.486071 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f0cfa-e81e-43a4-a908-366909696e7d None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:41.486358 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f0cfa-e81e-43a4-a908-366909696e7d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:41.486723 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f0cfa-e81e-43a4-a908-366909696e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:41.505580 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-e06f0cfa-e81e-43a4-a908-366909696e7d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:41.507127 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e06f0cfa-e81e-43a4-a908-366909696e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:41.535087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 78/165] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:26:41 2022] PUT /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785/users/be13617a9f5c4fd19704d8a065f5b59e/roles/917b4049bffd4db282dfe80ac37538be => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:26:41.536891 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69fb42-fdca-4ef1-bc90-7b5bd5010350 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:41.537393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69fb42-fdca-4ef1-bc90-7b5bd5010350 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:41.539235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69fb42-fdca-4ef1-bc90-7b5bd5010350 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:41.541131 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 79/166] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:41.552734 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a456ce5-14b2-4034-a683-86488c997f11 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:41.553372 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a456ce5-14b2-4034-a683-86488c997f11 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:41.553919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a456ce5-14b2-4034-a683-86488c997f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:41.565916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:41.582845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-7a456ce5-14b2-4034-a683-86488c997f11 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:41.584103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-7a456ce5-14b2-4034-a683-86488c997f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:41.670438 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 88/167] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:26:41.684181 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 80/168] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:41.713124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f0cfa-e81e-43a4-a908-366909696e7d None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:41.739051 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a456ce5-14b2-4034-a683-86488c997f11 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:41.885920 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a456ce5-14b2-4034-a683-86488c997f11 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:41.908810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f0cfa-e81e-43a4-a908-366909696e7d None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:41.912032 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:41.947167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9de69d-443d-4f74-8d0d-bdbdd3436d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:41.948966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9de69d-443d-4f74-8d0d-bdbdd3436d6d None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:41.949486 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9de69d-443d-4f74-8d0d-bdbdd3436d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:41.949978 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9de69d-443d-4f74-8d0d-bdbdd3436d6d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:41.967864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a125f3-d4cf-494f-b7eb-954b3fb457aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:41.968838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a125f3-d4cf-494f-b7eb-954b3fb457aa None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:41.969133 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a125f3-d4cf-494f-b7eb-954b3fb457aa None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:41.969413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a125f3-d4cf-494f-b7eb-954b3fb457aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:41.969820 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9de69d-443d-4f74-8d0d-bdbdd3436d6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:41.981553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-8d9de69d-443d-4f74-8d0d-bdbdd3436d6d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:26:41.981553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:41.981553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:41.981553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:41.981553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:41.981553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:41.981553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:41.981553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:41.981553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:41.981553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:41.981553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:41.981553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:41.981553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:41.981553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:41.981553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:41.981553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:41.982504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:41.982504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:41.982504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:41.982504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:41.982504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:41.982504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:41.982504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:41.982504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:41.982504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:41.982504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:41.982504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:41.982504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:41.982504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:41.982504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:41.982504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:41.982504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:41.983277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:41.983277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:41.983277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:41.983277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:41.983277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:41.983277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:41.983277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:41.983277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:41.983277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:41.983277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:41.983277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:26:41.983277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:26:41.985702 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 81/169] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:26:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:41.988716 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a125f3-d4cf-494f-b7eb-954b3fb457aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:41.991810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9de69d-443d-4f74-8d0d-bdbdd3436d6d None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:41.994179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-a9a125f3-d4cf-494f-b7eb-954b3fb457aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:41.994179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:41.994179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:41.994179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:41.994179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:41.994179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:41.994179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:41.994179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:41.994179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:41.994179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:41.994179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:41.994179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:41.994179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:41.994179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:41.994179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:41.994179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:41.995664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:41.995664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:41.995664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:41.995664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:41.995664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:41.995664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:41.995664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:41.995664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:41.995664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:41.995664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:41.995664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:41.995664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:41.995664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:41.995664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:41.995664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:41.995664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:41.996621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:41.996621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:41.996621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:41.996621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:41.996621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:41.996621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:41.996621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:41.996621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:41.996621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:41.996621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:41.996621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:41.996621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:26:42.001164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 89/170] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:42.006209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a125f3-d4cf-494f-b7eb-954b3fb457aa None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:42.074266 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:42.196511 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a125f3-d4cf-494f-b7eb-954b3fb457aa None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:42.226436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9de69d-443d-4f74-8d0d-bdbdd3436d6d None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:42.243544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479b260-6b58-4dce-80b5-759b794a3ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:42.244440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479b260-6b58-4dce-80b5-759b794a3ddc None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:42.244969 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479b260-6b58-4dce-80b5-759b794a3ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:42.245595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479b260-6b58-4dce-80b5-759b794a3ddc None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:42.246535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479b260-6b58-4dce-80b5-759b794a3ddc None admin] RBAC: Adding query filter params (name=member) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:42.247036 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479b260-6b58-4dce-80b5-759b794a3ddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:42.248636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479b260-6b58-4dce-80b5-759b794a3ddc None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:42.274376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 90/171] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:26:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:42.281714 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479b260-6b58-4dce-80b5-759b794a3ddc None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:42.293307 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c7e31-5b65-4652-ba8f-215ff49c02ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:42.294765 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c7e31-5b65-4652-ba8f-215ff49c02ac None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:42.295204 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c7e31-5b65-4652-ba8f-215ff49c02ac None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:42.295637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c7e31-5b65-4652-ba8f-215ff49c02ac None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:42.296549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c7e31-5b65-4652-ba8f-215ff49c02ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:42.297019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c7e31-5b65-4652-ba8f-215ff49c02ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:42.298589 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c7e31-5b65-4652-ba8f-215ff49c02ac None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:42.318907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 82/172] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:26:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:42.331898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c7e31-5b65-4652-ba8f-215ff49c02ac None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:42.418114 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:42.465380 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:42.484834 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479b260-6b58-4dce-80b5-759b794a3ddc None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:42.553506 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4618ee-2516-4edc-a949-7fde6c6de8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:42.554553 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4618ee-2516-4edc-a949-7fde6c6de8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:42.554850 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4618ee-2516-4edc-a949-7fde6c6de8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:42.555134 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4618ee-2516-4edc-a949-7fde6c6de8c2 None admin] PATH_INFO: `/v3/users/65e4eacde0544043820a4928b1ac4083` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:42.556607 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4618ee-2516-4edc-a949-7fde6c6de8c2 None admin] RBAC: Authorizing `identity:get_user(user_id=65e4eacde0544043820a4928b1ac4083)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:42.558506 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4618ee-2516-4edc-a949-7fde6c6de8c2 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:42.560370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 91/173] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:42 2022] GET /identity/v3/users/65e4eacde0544043820a4928b1ac4083 => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:42.560847 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c7e31-5b65-4652-ba8f-215ff49c02ac None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:42.565825 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4618ee-2516-4edc-a949-7fde6c6de8c2 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:42.651214 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6afb8-b842-42ff-a8af-e5de22a0dce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:42.652482 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6afb8-b842-42ff-a8af-e5de22a0dce7 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:42.652930 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6afb8-b842-42ff-a8af-e5de22a0dce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:42.653350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6afb8-b842-42ff-a8af-e5de22a0dce7 None admin] PATH_INFO: `/v3/users/65e4eacde0544043820a4928b1ac4083` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:42.657294 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6afb8-b842-42ff-a8af-e5de22a0dce7 None admin] RBAC: Authorizing `identity:get_user(user_id=65e4eacde0544043820a4928b1ac4083)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:42.658893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6afb8-b842-42ff-a8af-e5de22a0dce7 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:42.660722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 83/174] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:42 2022] GET /identity/v3/users/65e4eacde0544043820a4928b1ac4083 => generated 275 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:42.667236 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6afb8-b842-42ff-a8af-e5de22a0dce7 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:42.753747 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4618ee-2516-4edc-a949-7fde6c6de8c2 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:42.812004 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c153d0-9b4d-4cbc-88ec-2f4fc6c0a4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:42.812860 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c153d0-9b4d-4cbc-88ec-2f4fc6c0a4eb None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:42.813252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c153d0-9b4d-4cbc-88ec-2f4fc6c0a4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:42.813643 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c153d0-9b4d-4cbc-88ec-2f4fc6c0a4eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:42.857480 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:42.925670 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c153d0-9b4d-4cbc-88ec-2f4fc6c0a4eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:42.927523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c153d0-9b4d-4cbc-88ec-2f4fc6c0a4eb None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:42.970544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6afb8-b842-42ff-a8af-e5de22a0dce7 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:42.988939 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:43.049015 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 92/175] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 22:26:43.050416 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e909c65-db33-4664-966a-3c004924b5cf None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:43.055125 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e909c65-db33-4664-966a-3c004924b5cf None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:43.055605 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e909c65-db33-4664-966a-3c004924b5cf None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:43.057907 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 93/176] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:43.058794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e909c65-db33-4664-966a-3c004924b5cf None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:43.093852 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e8a60-1dac-41d2-b1a2-057d2964a0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:43.094523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e8a60-1dac-41d2-b1a2-057d2964a0fc None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:43.094808 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e8a60-1dac-41d2-b1a2-057d2964a0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:43.095085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e8a60-1dac-41d2-b1a2-057d2964a0fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:43.124808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:43.173417 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e909c65-db33-4664-966a-3c004924b5cf None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:43.237677 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda18a3-9a6c-406b-a219-a38d3dbb0b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:43.238557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda18a3-9a6c-406b-a219-a38d3dbb0b4a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:43.238841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda18a3-9a6c-406b-a219-a38d3dbb0b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:43.239118 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda18a3-9a6c-406b-a219-a38d3dbb0b4a None admin] PATH_INFO: `/v3/projects/fa88275401a14e5aaa627843dc619cc7` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:43.240861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda18a3-9a6c-406b-a219-a38d3dbb0b4a None admin] RBAC: Authorizing `identity:get_project(project_id=fa88275401a14e5aaa627843dc619cc7)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:43.242336 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda18a3-9a6c-406b-a219-a38d3dbb0b4a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:43.247647 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 94/177] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:43 2022] GET /identity/v3/projects/fa88275401a14e5aaa627843dc619cc7 => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:43.250684 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0645068-fefe-4136-918c-c378f8a7b67e None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:43.252125 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0645068-fefe-4136-918c-c378f8a7b67e None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:43.253477 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e8a60-1dac-41d2-b1a2-057d2964a0fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:43.255115 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e8a60-1dac-41d2-b1a2-057d2964a0fc None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:43.255554 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0645068-fefe-4136-918c-c378f8a7b67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:43.311718 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-a0645068-fefe-4136-918c-c378f8a7b67e None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:43.313108 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a0645068-fefe-4136-918c-c378f8a7b67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:43.392347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 84/178] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 726 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:26:43.393250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e8a60-1dac-41d2-b1a2-057d2964a0fc None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:43.397888 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:43.433579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 95/179] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:43.434733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0645068-fefe-4136-918c-c378f8a7b67e None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:43.516466 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:43.529443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e8a60-1dac-41d2-b1a2-057d2964a0fc None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:43.561637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0645068-fefe-4136-918c-c378f8a7b67e None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:43.565877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f736fb-1d46-4331-9ebd-890822e4d988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:43.566482 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f736fb-1d46-4331-9ebd-890822e4d988 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:43.566770 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f736fb-1d46-4331-9ebd-890822e4d988 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:43.567052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f736fb-1d46-4331-9ebd-890822e4d988 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:43.567900 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f736fb-1d46-4331-9ebd-890822e4d988 None admin] RBAC: Adding query filter params (role.id=1305747c745d49dc9a9718b7aa8aa4a9, scope.project.id=fa88275401a14e5aaa627843dc619cc7, user.id=65e4eacde0544043820a4928b1ac4083) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:43.568217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f736fb-1d46-4331-9ebd-890822e4d988 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:43.569703 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f736fb-1d46-4331-9ebd-890822e4d988 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:43.575675 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 85/180] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:43 2022] GET /identity/v3/role_assignments?user.id=65e4eacde0544043820a4928b1ac4083&scope.project.id=fa88275401a14e5aaa627843dc619cc7&role.id=1305747c745d49dc9a9718b7aa8aa4a9 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:43.577212 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a19e6-4b50-491a-ae12-c10bf367227a None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:43.577505 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a19e6-4b50-491a-ae12-c10bf367227a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:43.577784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a19e6-4b50-491a-ae12-c10bf367227a None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:43.579250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 86/181] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:43.581401 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a19e6-4b50-491a-ae12-c10bf367227a None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:43.621139 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e1cdc-1af6-4b40-b595-a33c9b045746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:43.622873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e1cdc-1af6-4b40-b595-a33c9b045746 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:43.623211 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e1cdc-1af6-4b40-b595-a33c9b045746 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:43.623498 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e1cdc-1af6-4b40-b595-a33c9b045746 None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:43.625495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e1cdc-1af6-4b40-b595-a33c9b045746 None admin] RBAC: Authorizing `identity:get_project(project_id=24e47f2dd0c74c218f4fd2059483f785)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:43.627502 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e1cdc-1af6-4b40-b595-a33c9b045746 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:43.630342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 96/182] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:43 2022] GET /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:43.631696 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a8c0d-9cbb-4e89-8902-97030f694cb0 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:43.631988 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a8c0d-9cbb-4e89-8902-97030f694cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:43.632273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a8c0d-9cbb-4e89-8902-97030f694cb0 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:43.633659 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 97/183] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:43.635475 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59515ddf-1671-426f-93b7-1c02a4a9cc89 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:43.635939 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59515ddf-1671-426f-93b7-1c02a4a9cc89 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:43.636382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59515ddf-1671-426f-93b7-1c02a4a9cc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:43.661530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-59515ddf-1671-426f-93b7-1c02a4a9cc89 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:43.663190 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-59515ddf-1671-426f-93b7-1c02a4a9cc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:43.683151 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:43.736070 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a19e6-4b50-491a-ae12-c10bf367227a None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:43.757629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 98/184] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:43.758795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59515ddf-1671-426f-93b7-1c02a4a9cc89 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:43.781664 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa04b2-c678-44fc-94c6-c59318aa082d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:43.782802 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa04b2-c678-44fc-94c6-c59318aa082d None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:43.783310 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa04b2-c678-44fc-94c6-c59318aa082d None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:43.783820 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa04b2-c678-44fc-94c6-c59318aa082d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:43.799794 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa04b2-c678-44fc-94c6-c59318aa082d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:43.808039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-e2fa04b2-c678-44fc-94c6-c59318aa082d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:43.808039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:43.808039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:43.808039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:43.808039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:43.808039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:43.808039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:43.808039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:43.808039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:43.808039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:43.808039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:43.808039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:43.808039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:43.808039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:43.808039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:43.808039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:43.809006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:43.809006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:43.809006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:43.809006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:43.809006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:43.809006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:43.809006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:43.809006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:43.809006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:43.809006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:43.809006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:43.809006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:43.809006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:43.809006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:43.809006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:43.809006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:43.810116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:43.810116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:43.810116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:43.810116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:43.810116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:43.810116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:43.810116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:43.810116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:43.810116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:43.810116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:43.810116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:43.810116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:26:43.810116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 87/185] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:26:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:43.810967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775594b6-9927-4cf1-92c2-915b3aed2e28 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:43.810967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775594b6-9927-4cf1-92c2-915b3aed2e28 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:43.811187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775594b6-9927-4cf1-92c2-915b3aed2e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:43.830450 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-775594b6-9927-4cf1-92c2-915b3aed2e28 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:43.831582 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-775594b6-9927-4cf1-92c2-915b3aed2e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:43.839990 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:43.908105 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59515ddf-1671-426f-93b7-1c02a4a9cc89 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:43.911032 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 88/186] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:43.911032 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775594b6-9927-4cf1-92c2-915b3aed2e28 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:43.954481 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19bc9b-81c8-4ae2-9fa4-bdc257ed2a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:43.955245 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19bc9b-81c8-4ae2-9fa4-bdc257ed2a02 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:43.955597 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19bc9b-81c8-4ae2-9fa4-bdc257ed2a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:43.955948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19bc9b-81c8-4ae2-9fa4-bdc257ed2a02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:43.956907 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19bc9b-81c8-4ae2-9fa4-bdc257ed2a02 None admin] RBAC: Adding query filter params (role.id=c3ad3cf6be504e35aaec10b7b9f3f34d, scope.project.id=24e47f2dd0c74c218f4fd2059483f785, user.id=65e4eacde0544043820a4928b1ac4083) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:43.957288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19bc9b-81c8-4ae2-9fa4-bdc257ed2a02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:43.958750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19bc9b-81c8-4ae2-9fa4-bdc257ed2a02 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:43.966157 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 99/187] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:43 2022] GET /identity/v3/role_assignments?user.id=65e4eacde0544043820a4928b1ac4083&scope.project.id=24e47f2dd0c74c218f4fd2059483f785&role.id=c3ad3cf6be504e35aaec10b7b9f3f34d => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:43.968854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19bc9b-81c8-4ae2-9fa4-bdc257ed2a02 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:43.986714 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:44.036535 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775594b6-9927-4cf1-92c2-915b3aed2e28 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:44.077278 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e61ea0-922a-4e5a-abad-98a80dbcaa57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:44.078522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e61ea0-922a-4e5a-abad-98a80dbcaa57 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:44.078995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e61ea0-922a-4e5a-abad-98a80dbcaa57 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:44.079451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e61ea0-922a-4e5a-abad-98a80dbcaa57 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:44.093148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e61ea0-922a-4e5a-abad-98a80dbcaa57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:44.102421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-61e61ea0-922a-4e5a-abad-98a80dbcaa57 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:26:44.102421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:44.102421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:44.102421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:44.102421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:44.102421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:44.102421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:44.102421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:44.102421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:44.102421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:44.102421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:44.102421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:44.102421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:44.102421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:44.102421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:44.102421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:44.103389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:44.103389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:44.103389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:44.103389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:44.103389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:44.103389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:44.103389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:44.103389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:44.103389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:44.103389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:44.103389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:44.103389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:44.103389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:44.103389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:44.103389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:44.103389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:44.104898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:44.104898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:44.104898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:44.104898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:44.104898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:44.104898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:44.104898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:44.104898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:44.104898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:44.104898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:44.104898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:26:44.104898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:26:44.104898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 89/188] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:26:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:44.106229 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e61ea0-922a-4e5a-abad-98a80dbcaa57 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:44.127418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19bc9b-81c8-4ae2-9fa4-bdc257ed2a02 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:44.151326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:44.175001 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1362d-472b-4c6a-b9d0-7e01b22f1232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:44.175687 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1362d-472b-4c6a-b9d0-7e01b22f1232 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:44.175975 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1362d-472b-4c6a-b9d0-7e01b22f1232 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:44.176313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1362d-472b-4c6a-b9d0-7e01b22f1232 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:44.177503 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1362d-472b-4c6a-b9d0-7e01b22f1232 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:44.177503 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1362d-472b-4c6a-b9d0-7e01b22f1232 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:44.178945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1362d-472b-4c6a-b9d0-7e01b22f1232 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:44.189493 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 100/189] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:44.190241 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1362d-472b-4c6a-b9d0-7e01b22f1232 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:44.231534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e61ea0-922a-4e5a-abad-98a80dbcaa57 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:44.266894 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:44.289866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f22647-453f-43d4-8d30-831ad21c9eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:44.290681 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f22647-453f-43d4-8d30-831ad21c9eaf None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:44.290973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f22647-453f-43d4-8d30-831ad21c9eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:44.291293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f22647-453f-43d4-8d30-831ad21c9eaf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:44.292508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f22647-453f-43d4-8d30-831ad21c9eaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:44.294752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f22647-453f-43d4-8d30-831ad21c9eaf None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:44.297189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 90/190] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:44.297950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f22647-453f-43d4-8d30-831ad21c9eaf None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:44.308752 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1362d-472b-4c6a-b9d0-7e01b22f1232 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:44.343581 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7024391-bde4-46a5-a476-46c9bbbff346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:44.344181 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7024391-bde4-46a5-a476-46c9bbbff346 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:44.344470 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7024391-bde4-46a5-a476-46c9bbbff346 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:44.344783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7024391-bde4-46a5-a476-46c9bbbff346 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:44.345538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7024391-bde4-46a5-a476-46c9bbbff346 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:44.345879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7024391-bde4-46a5-a476-46c9bbbff346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:44.347253 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7024391-bde4-46a5-a476-46c9bbbff346 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:44.355298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 101/191] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:26:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:44.356658 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d5967-426a-49ae-b1a0-3b81f990466a None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:44.356951 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d5967-426a-49ae-b1a0-3b81f990466a None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:44.357235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d5967-426a-49ae-b1a0-3b81f990466a None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:44.358587 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 102/192] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:44.359201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d5967-426a-49ae-b1a0-3b81f990466a None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:44.407071 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:44.433388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f22647-453f-43d4-8d30-831ad21c9eaf None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:44.466182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d5967-426a-49ae-b1a0-3b81f990466a None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:44.466739 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e89865-affc-4b40-8a27-c225ecc0b950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:44.467628 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e89865-affc-4b40-8a27-c225ecc0b950 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:44.467915 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e89865-affc-4b40-8a27-c225ecc0b950 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:44.468209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e89865-affc-4b40-8a27-c225ecc0b950 None admin] PATH_INFO: `/v3/users/8aec0c782060448fbf49b7cadf083a17` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:44.481835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e89865-affc-4b40-8a27-c225ecc0b950 None admin] RBAC: Authorizing `identity:get_user(user_id=8aec0c782060448fbf49b7cadf083a17)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:44.483291 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e89865-affc-4b40-8a27-c225ecc0b950 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:44.484996 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 91/193] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:44 2022] GET /identity/v3/users/8aec0c782060448fbf49b7cadf083a17 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:44.490122 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e89865-affc-4b40-8a27-c225ecc0b950 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:44.499529 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee2c57-6d9d-4eb6-a79f-c9d45bd808c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:44.500196 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee2c57-6d9d-4eb6-a79f-c9d45bd808c6 None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:44.500476 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee2c57-6d9d-4eb6-a79f-c9d45bd808c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:44.500790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee2c57-6d9d-4eb6-a79f-c9d45bd808c6 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:44.501519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee2c57-6d9d-4eb6-a79f-c9d45bd808c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:44.502942 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee2c57-6d9d-4eb6-a79f-c9d45bd808c6 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:44.534371 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:44.586359 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e89865-affc-4b40-8a27-c225ecc0b950 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:44.602573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 103/194] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 22:26:44 2022] POST /identity/v3/users => generated 297 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:26:44.602573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abd0f8-a635-4f2f-a23d-b92e06a6e6f4 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:44.602573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abd0f8-a635-4f2f-a23d-b92e06a6e6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:44.602573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abd0f8-a635-4f2f-a23d-b92e06a6e6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:44.617227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-14abd0f8-a635-4f2f-a23d-b92e06a6e6f4 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:44.618507 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-14abd0f8-a635-4f2f-a23d-b92e06a6e6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:44.636699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18bbf2c-afda-407e-8368-45af2536bfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:44.637595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bbf2c-afda-407e-8368-45af2536bfeb None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:44.637882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bbf2c-afda-407e-8368-45af2536bfeb None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:44.638160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bbf2c-afda-407e-8368-45af2536bfeb None admin] PATH_INFO: `/v3/users/8aec0c782060448fbf49b7cadf083a17` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:44.639393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18bbf2c-afda-407e-8368-45af2536bfeb None admin] RBAC: Authorizing `identity:get_user(user_id=8aec0c782060448fbf49b7cadf083a17)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:44.641061 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18bbf2c-afda-407e-8368-45af2536bfeb None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:44.644755 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 92/195] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:44 2022] GET /identity/v3/users/8aec0c782060448fbf49b7cadf083a17 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:44.646216 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18bbf2c-afda-407e-8368-45af2536bfeb None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:44.695527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 104/196] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:44.696719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abd0f8-a635-4f2f-a23d-b92e06a6e6f4 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:44.730627 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:44.776649 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18bbf2c-afda-407e-8368-45af2536bfeb None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:44.792367 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:44.816250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abd0f8-a635-4f2f-a23d-b92e06a6e6f4 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:44.817380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597957ea-faa7-463e-bb84-2f67dfea5e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:44.818105 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597957ea-faa7-463e-bb84-2f67dfea5e0f None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:44.818839 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597957ea-faa7-463e-bb84-2f67dfea5e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:44.819736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597957ea-faa7-463e-bb84-2f67dfea5e0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:44.863455 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4855f-797e-4504-86de-29f80b4eb1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:44.864185 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4855f-797e-4504-86de-29f80b4eb1ee None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:44.864474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4855f-797e-4504-86de-29f80b4eb1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:44.864787 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4855f-797e-4504-86de-29f80b4eb1ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:44.912157 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597957ea-faa7-463e-bb84-2f67dfea5e0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:44.914063 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597957ea-faa7-463e-bb84-2f67dfea5e0f None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:44.952599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be4855f-797e-4504-86de-29f80b4eb1ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:44.955267 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be4855f-797e-4504-86de-29f80b4eb1ee None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:44.983597 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:45.055218 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 93/197] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:26:45.056258 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597957ea-faa7-463e-bb84-2f67dfea5e0f None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:45.073455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:45.083349 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 105/198] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:26:45.084545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4855f-797e-4504-86de-29f80b4eb1ee None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:45.191751 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597957ea-faa7-463e-bb84-2f67dfea5e0f None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:45.199465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4855f-797e-4504-86de-29f80b4eb1ee None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:45.227552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:45.236501 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3ba43-4c17-4304-8dcf-a902446dd297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:45.237596 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3ba43-4c17-4304-8dcf-a902446dd297 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:45.237997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3ba43-4c17-4304-8dcf-a902446dd297 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:45.238430 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3ba43-4c17-4304-8dcf-a902446dd297 None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:45.239638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3ba43-4c17-4304-8dcf-a902446dd297 None admin] RBAC: Authorizing `identity:get_project(project_id=debcfc4c17a34d8a80f13016124abfaf)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:45.239933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3852cf-7077-403f-b650-3c9f0c190b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:45.240844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3852cf-7077-403f-b650-3c9f0c190b8b None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:45.241254 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3852cf-7077-403f-b650-3c9f0c190b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:45.241409 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3ba43-4c17-4304-8dcf-a902446dd297 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:45.241674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3852cf-7077-403f-b650-3c9f0c190b8b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:45.243271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 106/199] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:45 2022] GET /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:45.244651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3ba43-4c17-4304-8dcf-a902446dd297 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:45.256156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3852cf-7077-403f-b650-3c9f0c190b8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:45.267462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-2a3852cf-7077-403f-b650-3c9f0c190b8b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:26:45.267462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:45.267462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:45.267462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:45.267462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:45.267462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:45.267462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:45.267462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:45.267462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:45.267462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:45.267462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:45.267462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:45.267462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:45.267462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:45.267462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:45.267462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:45.269877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:45.269877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:45.269877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:45.269877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:45.269877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:45.269877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:45.269877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:45.269877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:45.269877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:45.269877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:45.269877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:45.269877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:45.269877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:45.269877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:45.269877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:45.269877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:45.271376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:45.271376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:45.271376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:45.271376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:45.271376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:45.271376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:45.271376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:45.271376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:45.271376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:45.271376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:45.271376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:26:45.271376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:26:45.271376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 94/200] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:45.271376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3852cf-7077-403f-b650-3c9f0c190b8b None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:45.357979 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3ba43-4c17-4304-8dcf-a902446dd297 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:45.364638 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:45.396498 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c78c4-8a36-428a-a431-c9f81d2992e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:45.397789 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c78c4-8a36-428a-a431-c9f81d2992e8 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:45.398275 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c78c4-8a36-428a-a431-c9f81d2992e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:45.398751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c78c4-8a36-428a-a431-c9f81d2992e8 None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:45.400181 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c78c4-8a36-428a-a431-c9f81d2992e8 None admin] RBAC: Authorizing `identity:get_project(project_id=debcfc4c17a34d8a80f13016124abfaf)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:45.402398 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c78c4-8a36-428a-a431-c9f81d2992e8 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:45.403601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3852cf-7077-403f-b650-3c9f0c190b8b None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:45.404766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 107/201] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:45 2022] GET /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:45.405912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c78c4-8a36-428a-a431-c9f81d2992e8 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:45.451251 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c4a4c-d879-485a-82d1-86ccb193359f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:45.451962 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c4a4c-d879-485a-82d1-86ccb193359f None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:45.452132 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c4a4c-d879-485a-82d1-86ccb193359f None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:45.452400 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c4a4c-d879-485a-82d1-86ccb193359f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:45.453853 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c4a4c-d879-485a-82d1-86ccb193359f None admin] RBAC: Adding query filter params (role.id=6ff7708b59984b9287cc81d46a57b4b4, scope.project.id=debcfc4c17a34d8a80f13016124abfaf, user.id=8aec0c782060448fbf49b7cadf083a17) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:45.454178 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c4a4c-d879-485a-82d1-86ccb193359f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:45.455918 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c4a4c-d879-485a-82d1-86ccb193359f None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:45.464393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 95/202] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:45 2022] GET /identity/v3/role_assignments?user.id=8aec0c782060448fbf49b7cadf083a17&scope.project.id=debcfc4c17a34d8a80f13016124abfaf&role.id=6ff7708b59984b9287cc81d46a57b4b4 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:45.465127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c4a4c-d879-485a-82d1-86ccb193359f None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:45.477373 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:45.523258 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c78c4-8a36-428a-a431-c9f81d2992e8 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:45.562730 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091d5dc-c085-4500-b7e6-4c6b131e3c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:45.563783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091d5dc-c085-4500-b7e6-4c6b131e3c47 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:45.564272 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091d5dc-c085-4500-b7e6-4c6b131e3c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:45.564790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091d5dc-c085-4500-b7e6-4c6b131e3c47 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:45.565787 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091d5dc-c085-4500-b7e6-4c6b131e3c47 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:45.566336 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091d5dc-c085-4500-b7e6-4c6b131e3c47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:45.567972 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091d5dc-c085-4500-b7e6-4c6b131e3c47 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:45.578801 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 108/203] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:26:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:45.587913 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091d5dc-c085-4500-b7e6-4c6b131e3c47 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:45.613399 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c4a4c-d879-485a-82d1-86ccb193359f None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:45.658589 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cf212-1661-4055-8e92-57616d8d15b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:45.659272 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cf212-1661-4055-8e92-57616d8d15b5 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:45.659563 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cf212-1661-4055-8e92-57616d8d15b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:45.659846 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cf212-1661-4055-8e92-57616d8d15b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:45.661263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693cf212-1661-4055-8e92-57616d8d15b5 None admin] RBAC: Adding query filter params (role.id=c3ad3cf6be504e35aaec10b7b9f3f34d, scope.project.id=debcfc4c17a34d8a80f13016124abfaf, user.id=8aec0c782060448fbf49b7cadf083a17) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:45.661591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693cf212-1661-4055-8e92-57616d8d15b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:45.663387 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693cf212-1661-4055-8e92-57616d8d15b5 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:45.667098 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:45.671715 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 96/204] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:45 2022] GET /identity/v3/role_assignments?user.id=8aec0c782060448fbf49b7cadf083a17&scope.project.id=debcfc4c17a34d8a80f13016124abfaf&role.id=c3ad3cf6be504e35aaec10b7b9f3f34d => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:45.751430 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091d5dc-c085-4500-b7e6-4c6b131e3c47 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:45.779025 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:45.796604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9f1bf-699d-41d0-bcab-0cb387042bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:45.797524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9f1bf-699d-41d0-bcab-0cb387042bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:45.797817 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9f1bf-699d-41d0-bcab-0cb387042bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:45.798098 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9f1bf-699d-41d0-bcab-0cb387042bc0 None admin] PATH_INFO: `/v3/users/be13617a9f5c4fd19704d8a065f5b59e` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:45.808058 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9f1bf-699d-41d0-bcab-0cb387042bc0 None admin] RBAC: Authorizing `identity:get_user(user_id=be13617a9f5c4fd19704d8a065f5b59e)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:45.808058 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9f1bf-699d-41d0-bcab-0cb387042bc0 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:45.808058 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 109/205] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:45 2022] GET /identity/v3/users/be13617a9f5c4fd19704d8a065f5b59e => generated 289 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:45.808849 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cf212-1661-4055-8e92-57616d8d15b5 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:45.922856 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cf212-1661-4055-8e92-57616d8d15b5 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:45.964855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4233308-abbd-40b6-9818-cc8377468b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:45.965524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4233308-abbd-40b6-9818-cc8377468b19 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:45.965870 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4233308-abbd-40b6-9818-cc8377468b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:45.966162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4233308-abbd-40b6-9818-cc8377468b19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:46.035750 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:46.038351 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4233308-abbd-40b6-9818-cc8377468b19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:46.039856 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4233308-abbd-40b6-9818-cc8377468b19 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:46.144629 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 97/206] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:26:46.150390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9f1bf-699d-41d0-bcab-0cb387042bc0 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:46.265129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9f1bf-699d-41d0-bcab-0cb387042bc0 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:46.308785 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fd8e8-4a20-4ddb-ae4e-da508d7601c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:46.309678 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fd8e8-4a20-4ddb-ae4e-da508d7601c2 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:46.310014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fd8e8-4a20-4ddb-ae4e-da508d7601c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:46.310353 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fd8e8-4a20-4ddb-ae4e-da508d7601c2 None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:46.311595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7fd8e8-4a20-4ddb-ae4e-da508d7601c2 None admin] RBAC: Authorizing `identity:get_project(project_id=24e47f2dd0c74c218f4fd2059483f785)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:46.318356 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7fd8e8-4a20-4ddb-ae4e-da508d7601c2 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:46.318356 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 110/207] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:46 2022] GET /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:46.320653 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4233308-abbd-40b6-9818-cc8377468b19 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:46.432700 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:46.432700 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4233308-abbd-40b6-9818-cc8377468b19 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:46.520606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd03858-4819-4a5c-b2b2-22c65931a5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:46.521290 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd03858-4819-4a5c-b2b2-22c65931a5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:46.524595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd03858-4819-4a5c-b2b2-22c65931a5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:46.524595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd03858-4819-4a5c-b2b2-22c65931a5a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:46.524595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd03858-4819-4a5c-b2b2-22c65931a5a7 None admin] RBAC: Adding query filter params (role.id=917b4049bffd4db282dfe80ac37538be, scope.project.id=24e47f2dd0c74c218f4fd2059483f785, user.id=be13617a9f5c4fd19704d8a065f5b59e) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:46.524595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd03858-4819-4a5c-b2b2-22c65931a5a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:46.524595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd03858-4819-4a5c-b2b2-22c65931a5a7 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:46.544726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 98/208] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:46 2022] GET /identity/v3/role_assignments?user.id=be13617a9f5c4fd19704d8a065f5b59e&scope.project.id=24e47f2dd0c74c218f4fd2059483f785&role.id=917b4049bffd4db282dfe80ac37538be => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:47.109025 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1058e-61af-4b5e-85bb-257bc4548743 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:47.109025 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1058e-61af-4b5e-85bb-257bc4548743 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:47.109025 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1058e-61af-4b5e-85bb-257bc4548743 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:47.109991 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 111/209] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:47.114463 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45e08a-7680-4568-9ab9-c8bb6dfc9b75 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:47.114760 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45e08a-7680-4568-9ab9-c8bb6dfc9b75 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:47.115045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45e08a-7680-4568-9ab9-c8bb6dfc9b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:47.136203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-2f45e08a-7680-4568-9ab9-c8bb6dfc9b75 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:47.137495 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-2f45e08a-7680-4568-9ab9-c8bb6dfc9b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:47.234116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 99/210] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:47.258878 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1058e-61af-4b5e-85bb-257bc4548743 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:47.329050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c3f03-32ce-46ca-b0a1-e241a91e9074 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:47.329390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c3f03-32ce-46ca-b0a1-e241a91e9074 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:47.329622 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c3f03-32ce-46ca-b0a1-e241a91e9074 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:47.331001 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 100/211] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:47.335960 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668315c3-8abf-41d9-aabe-0cb44c382d85 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:47.336255 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668315c3-8abf-41d9-aabe-0cb44c382d85 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:47.336539 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668315c3-8abf-41d9-aabe-0cb44c382d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:47.338175 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:47.396259 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-668315c3-8abf-41d9-aabe-0cb44c382d85 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:47.397626 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-668315c3-8abf-41d9-aabe-0cb44c382d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:47.416862 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1058e-61af-4b5e-85bb-257bc4548743 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:47.451370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f62b5-016d-418a-8fab-5738bb8acb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:47.452173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f62b5-016d-418a-8fab-5738bb8acb6f None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:47.452458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f62b5-016d-418a-8fab-5738bb8acb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:47.452764 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f62b5-016d-418a-8fab-5738bb8acb6f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:47.463326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f62b5-016d-418a-8fab-5738bb8acb6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:47.471262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-151f62b5-016d-418a-8fab-5738bb8acb6f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:47.471262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:47.471262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:47.471262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:47.471262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:47.471262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:47.471262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:47.471262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:47.471262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:47.471262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:47.471262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:47.471262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:47.471262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:47.471262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:47.471262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:47.471262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:47.473847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:47.473847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:47.473847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:47.473847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:47.473847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:47.473847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:47.473847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:47.473847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:47.473847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:47.473847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:47.473847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:47.473847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:47.473847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:47.473847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:47.473847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:47.473847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:47.475822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:47.475822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:47.475822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:47.475822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:47.475822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:47.475822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:47.475822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:47.475822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:47.475822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:47.475822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:47.475822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:47.475822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:26:47.475822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 112/212] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:47.477502 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 101/213] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:47.479728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f62b5-016d-418a-8fab-5738bb8acb6f None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:47.505608 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668315c3-8abf-41d9-aabe-0cb44c382d85 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:47.533022 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:47.597672 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f62b5-016d-418a-8fab-5738bb8acb6f None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:47.611500 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668315c3-8abf-41d9-aabe-0cb44c382d85 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:47.620955 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:47.636054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aac10c-184b-41f3-a00d-00f2159e6689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:47.637132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aac10c-184b-41f3-a00d-00f2159e6689 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:47.637628 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aac10c-184b-41f3-a00d-00f2159e6689 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:47.638110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aac10c-184b-41f3-a00d-00f2159e6689 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:47.639093 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aac10c-184b-41f3-a00d-00f2159e6689 None admin] RBAC: Adding query filter params (name=member) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:47.639605 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aac10c-184b-41f3-a00d-00f2159e6689 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:47.641251 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aac10c-184b-41f3-a00d-00f2159e6689 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:47.651391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 113/214] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:26:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:47.663327 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80b0b8-b50d-4fa4-aa08-6fe4a695cbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:47.663502 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aac10c-184b-41f3-a00d-00f2159e6689 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:47.663581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80b0b8-b50d-4fa4-aa08-6fe4a695cbe2 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:47.663581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80b0b8-b50d-4fa4-aa08-6fe4a695cbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:47.663581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80b0b8-b50d-4fa4-aa08-6fe4a695cbe2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:47.663581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80b0b8-b50d-4fa4-aa08-6fe4a695cbe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:47.663581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80b0b8-b50d-4fa4-aa08-6fe4a695cbe2 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:47.663581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 102/215] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:47.669509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80b0b8-b50d-4fa4-aa08-6fe4a695cbe2 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:47.764858 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:47.767937 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aac10c-184b-41f3-a00d-00f2159e6689 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:47.774090 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80b0b8-b50d-4fa4-aa08-6fe4a695cbe2 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:47.806284 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf741f0-adc1-4fc7-996c-7629dafb8231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:47.808290 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf741f0-adc1-4fc7-996c-7629dafb8231 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:47.808700 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf741f0-adc1-4fc7-996c-7629dafb8231 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:47.809153 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf741f0-adc1-4fc7-996c-7629dafb8231 None admin] PATH_INFO: `/v3/users/8ceda92aea5c4204a152f074a9e0d28d` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:47.810531 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0ac7f-a340-4306-9a7f-83ff8dad440f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:47.811203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0ac7f-a340-4306-9a7f-83ff8dad440f None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:47.811487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0ac7f-a340-4306-9a7f-83ff8dad440f None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:47.811762 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0ac7f-a340-4306-9a7f-83ff8dad440f None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:47.812490 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b0ac7f-a340-4306-9a7f-83ff8dad440f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:47.814021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b0ac7f-a340-4306-9a7f-83ff8dad440f None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:47.826625 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf741f0-adc1-4fc7-996c-7629dafb8231 None admin] RBAC: Authorizing `identity:get_user(user_id=8ceda92aea5c4204a152f074a9e0d28d)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:47.828129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf741f0-adc1-4fc7-996c-7629dafb8231 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:47.830068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 114/216] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:47 2022] GET /identity/v3/users/8ceda92aea5c4204a152f074a9e0d28d => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:47.835564 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf741f0-adc1-4fc7-996c-7629dafb8231 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:47.880239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:47.920548 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 103/217] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 22:26:47 2022] POST /identity/v3/users => generated 297 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:26:47.950725 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf741f0-adc1-4fc7-996c-7629dafb8231 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:47.991182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74cb53-66a3-4b68-a9ed-d1ad27796969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:47.991865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74cb53-66a3-4b68-a9ed-d1ad27796969 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:47.992156 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74cb53-66a3-4b68-a9ed-d1ad27796969 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:47.992441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74cb53-66a3-4b68-a9ed-d1ad27796969 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:48.053581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3486631-5ccf-4ce9-b727-8c568d5eb92b None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:48.054047 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3486631-5ccf-4ce9-b727-8c568d5eb92b None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:48.054330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3486631-5ccf-4ce9-b727-8c568d5eb92b None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:48.056130 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 104/218] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:26:48.062431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c0ab3-fa01-4047-9fdd-ca788145993a None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:48.063004 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c0ab3-fa01-4047-9fdd-ca788145993a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:48.063385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c0ab3-fa01-4047-9fdd-ca788145993a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:48.073959 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74cb53-66a3-4b68-a9ed-d1ad27796969 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:48.075696 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74cb53-66a3-4b68-a9ed-d1ad27796969 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:48.087108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-ae7c0ab3-fa01-4047-9fdd-ca788145993a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:48.089401 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ae7c0ab3-fa01-4047-9fdd-ca788145993a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:48.134254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:48.179932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 105/219] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 23 22:26:48.181180 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb617a61-1b78-468a-afdd-3d42c961cb06 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:48.181483 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb617a61-1b78-468a-afdd-3d42c961cb06 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:48.181859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb617a61-1b78-468a-afdd-3d42c961cb06 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:48.183199 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 106/220] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:48.190272 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faf1ad-1232-4f3c-8768-4062114177ec None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:48.190609 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faf1ad-1232-4f3c-8768-4062114177ec None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:48.190916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faf1ad-1232-4f3c-8768-4062114177ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:48.205856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 115/221] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 22:26:48.205856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74cb53-66a3-4b68-a9ed-d1ad27796969 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:48.210199 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-13faf1ad-1232-4f3c-8768-4062114177ec None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:48.210702 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-13faf1ad-1232-4f3c-8768-4062114177ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:48.292660 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 107/222] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:26:48.293757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faf1ad-1232-4f3c-8768-4062114177ec None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:48.304616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:48.335478 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74cb53-66a3-4b68-a9ed-d1ad27796969 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:48.376148 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb6124-d368-4966-a581-51dbb997e3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:48.377056 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb6124-d368-4966-a581-51dbb997e3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:48.377343 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb6124-d368-4966-a581-51dbb997e3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:48.377622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb6124-d368-4966-a581-51dbb997e3e1 None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:48.378867 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb6124-d368-4966-a581-51dbb997e3e1 None admin] RBAC: Authorizing `identity:get_project(project_id=debcfc4c17a34d8a80f13016124abfaf)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:48.380311 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb6124-d368-4966-a581-51dbb997e3e1 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:48.384657 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 116/223] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:48 2022] GET /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:48.384657 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb6124-d368-4966-a581-51dbb997e3e1 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:48.395839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:48.426126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faf1ad-1232-4f3c-8768-4062114177ec None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:48.470122 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f110da-dfe8-4709-a219-27c817fcfdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:48.471650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f110da-dfe8-4709-a219-27c817fcfdcc None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:48.472509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f110da-dfe8-4709-a219-27c817fcfdcc None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:48.473140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f110da-dfe8-4709-a219-27c817fcfdcc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:48.487874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f110da-dfe8-4709-a219-27c817fcfdcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:48.497422 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb6124-d368-4966-a581-51dbb997e3e1 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:48.498732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-49f110da-dfe8-4709-a219-27c817fcfdcc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:48.498732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:48.498732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:48.498732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:48.498732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:48.498732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:48.498732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:48.498732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:48.498732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:48.498732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:48.498732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:48.498732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:48.498732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:48.498732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:48.498732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:48.498732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:48.499601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:48.499601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:48.499601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:48.499601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:48.499601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:48.499601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:48.499601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:48.499601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:48.499601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:48.499601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:48.499601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:48.499601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:48.499601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:48.499601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:48.499601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:48.499601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:48.500364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:48.500364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:48.500364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:48.500364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:48.500364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:48.500364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:48.500364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:48.500364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:48.500364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:48.500364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:48.500364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:48.500364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:26:48.503711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 108/224] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:26:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:48.504683 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f110da-dfe8-4709-a219-27c817fcfdcc None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:48.541123 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56165b61-0208-4db6-8bb6-daf66860e069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:48.542152 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56165b61-0208-4db6-8bb6-daf66860e069 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:48.542535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56165b61-0208-4db6-8bb6-daf66860e069 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:48.542911 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56165b61-0208-4db6-8bb6-daf66860e069 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:48.554546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56165b61-0208-4db6-8bb6-daf66860e069 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:48.563934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-56165b61-0208-4db6-8bb6-daf66860e069 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:26:48.563934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:48.563934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:48.563934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:48.563934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:48.563934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:48.563934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:48.563934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:48.563934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:48.563934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:48.563934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:48.563934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:48.563934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:48.563934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:48.563934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:48.563934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:48.565049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:48.565049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:48.565049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:48.565049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:48.565049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:48.565049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:48.565049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:48.565049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:48.565049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:48.565049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:48.565049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:48.565049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:48.565049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:48.565049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:48.565049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:48.565049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:48.566029 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:48.566029 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:48.566029 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:48.566029 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:48.566029 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:48.566029 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:48.566029 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:48.566029 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:48.566029 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:48.566029 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:48.566029 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:26:48.566029 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:26:48.567903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 117/225] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:26:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:48.568652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56165b61-0208-4db6-8bb6-daf66860e069 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:48.605756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:48.638707 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f110da-dfe8-4709-a219-27c817fcfdcc None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:48.644776 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:48.679369 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456b19e-6496-4580-8bfd-d96d902e61c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:48.680042 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456b19e-6496-4580-8bfd-d96d902e61c3 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:48.680329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456b19e-6496-4580-8bfd-d96d902e61c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:48.680770 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456b19e-6496-4580-8bfd-d96d902e61c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:48.682357 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456b19e-6496-4580-8bfd-d96d902e61c3 None admin] RBAC: Adding query filter params (role.id=1305747c745d49dc9a9718b7aa8aa4a9, scope.project.id=debcfc4c17a34d8a80f13016124abfaf, user.id=8ceda92aea5c4204a152f074a9e0d28d) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:48.682652 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456b19e-6496-4580-8bfd-d96d902e61c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:48.684610 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456b19e-6496-4580-8bfd-d96d902e61c3 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:48.687310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56165b61-0208-4db6-8bb6-daf66860e069 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:48.690784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 109/226] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:48 2022] GET /identity/v3/role_assignments?user.id=8ceda92aea5c4204a152f074a9e0d28d&scope.project.id=debcfc4c17a34d8a80f13016124abfaf&role.id=1305747c745d49dc9a9718b7aa8aa4a9 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:48.692620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456b19e-6496-4580-8bfd-d96d902e61c3 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:48.730041 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5085afd-0281-49c2-a4a5-32c76fd2657c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:48.734437 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5085afd-0281-49c2-a4a5-32c76fd2657c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:48.734437 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5085afd-0281-49c2-a4a5-32c76fd2657c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:48.734437 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5085afd-0281-49c2-a4a5-32c76fd2657c None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:48.734747 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5085afd-0281-49c2-a4a5-32c76fd2657c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:48.735036 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5085afd-0281-49c2-a4a5-32c76fd2657c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:48.736461 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5085afd-0281-49c2-a4a5-32c76fd2657c None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:48.746683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 118/227] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:48.756324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5085afd-0281-49c2-a4a5-32c76fd2657c None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:48.893878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456b19e-6496-4580-8bfd-d96d902e61c3 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:48.926473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:48.933203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e57ff-4bf2-44e3-8b6a-354d4791ab64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:48.934056 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e57ff-4bf2-44e3-8b6a-354d4791ab64 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:48.934515 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e57ff-4bf2-44e3-8b6a-354d4791ab64 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:48.934933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e57ff-4bf2-44e3-8b6a-354d4791ab64 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:48.935824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e57ff-4bf2-44e3-8b6a-354d4791ab64 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:48.936273 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e57ff-4bf2-44e3-8b6a-354d4791ab64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:48.937833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e57ff-4bf2-44e3-8b6a-354d4791ab64 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:48.947897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 110/228] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:26:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:48.956693 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5085afd-0281-49c2-a4a5-32c76fd2657c None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:48.958162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e57ff-4bf2-44e3-8b6a-354d4791ab64 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:48.997736 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:49.018195 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e621d-f3ee-4c40-afcf-2d37751d6137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:49.019096 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e621d-f3ee-4c40-afcf-2d37751d6137 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:49.019394 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e621d-f3ee-4c40-afcf-2d37751d6137 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:49.019677 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e621d-f3ee-4c40-afcf-2d37751d6137 None admin] PATH_INFO: `/v3/users/8aec0c782060448fbf49b7cadf083a17` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:49.025682 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e621d-f3ee-4c40-afcf-2d37751d6137 None admin] RBAC: Authorizing `identity:get_user(user_id=8aec0c782060448fbf49b7cadf083a17)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:49.030156 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e621d-f3ee-4c40-afcf-2d37751d6137 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:49.032228 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 119/229] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:48 2022] GET /identity/v3/users/8aec0c782060448fbf49b7cadf083a17 => generated 283 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:49.042053 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e621d-f3ee-4c40-afcf-2d37751d6137 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:49.067937 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e57ff-4bf2-44e3-8b6a-354d4791ab64 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:49.104683 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684cf44-5406-4315-af19-5389783f6de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:49.106124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684cf44-5406-4315-af19-5389783f6de6 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:49.106570 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684cf44-5406-4315-af19-5389783f6de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:49.106984 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684cf44-5406-4315-af19-5389783f6de6 None admin] PATH_INFO: `/v3/users/8aec0c782060448fbf49b7cadf083a17` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:49.108467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c684cf44-5406-4315-af19-5389783f6de6 None admin] RBAC: Authorizing `identity:get_user(user_id=8aec0c782060448fbf49b7cadf083a17)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:49.110092 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c684cf44-5406-4315-af19-5389783f6de6 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:49.112200 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 111/230] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:48 2022] GET /identity/v3/users/8aec0c782060448fbf49b7cadf083a17 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:49.117339 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684cf44-5406-4315-af19-5389783f6de6 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:49.183547 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:49.226576 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684cf44-5406-4315-af19-5389783f6de6 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:49.249497 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e621d-f3ee-4c40-afcf-2d37751d6137 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:49.267007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68ad32-deb1-4b45-b0db-2ea92f984558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:49.268331 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68ad32-deb1-4b45-b0db-2ea92f984558 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:49.268846 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68ad32-deb1-4b45-b0db-2ea92f984558 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:49.270538 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68ad32-deb1-4b45-b0db-2ea92f984558 None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:49.272016 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68ad32-deb1-4b45-b0db-2ea92f984558 None admin] RBAC: Authorizing `identity:get_project(project_id=debcfc4c17a34d8a80f13016124abfaf)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:49.274645 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68ad32-deb1-4b45-b0db-2ea92f984558 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:49.280938 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 112/231] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:49 2022] GET /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:49.285654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68ad32-deb1-4b45-b0db-2ea92f984558 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:49.341982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee8168-6ca5-4f0d-925a-62865990cfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:49.342906 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee8168-6ca5-4f0d-925a-62865990cfe9 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:49.343195 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee8168-6ca5-4f0d-925a-62865990cfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:49.343476 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee8168-6ca5-4f0d-925a-62865990cfe9 None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:49.344820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee8168-6ca5-4f0d-925a-62865990cfe9 None admin] RBAC: Authorizing `identity:get_project(project_id=debcfc4c17a34d8a80f13016124abfaf)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:49.346304 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee8168-6ca5-4f0d-925a-62865990cfe9 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:49.348093 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 120/232] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:49 2022] GET /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:49.357742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee8168-6ca5-4f0d-925a-62865990cfe9 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:49.485590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:49.490462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68ad32-deb1-4b45-b0db-2ea92f984558 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:49.532047 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee8168-6ca5-4f0d-925a-62865990cfe9 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:49.566013 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:49.573881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b488572-e82a-4010-875c-dca978d0386a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:49.574806 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b488572-e82a-4010-875c-dca978d0386a None admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:49.575094 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b488572-e82a-4010-875c-dca978d0386a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:49.575382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b488572-e82a-4010-875c-dca978d0386a None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf/users/8aec0c782060448fbf49b7cadf083a17/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:49.577734 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5a870-8e88-44d2-984d-7197b15b0e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:49.578633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5a870-8e88-44d2-984d-7197b15b0e2d None admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:49.578919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5a870-8e88-44d2-984d-7197b15b0e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:49.579203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5a870-8e88-44d2-984d-7197b15b0e2d None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf/users/8aec0c782060448fbf49b7cadf083a17/roles/c3ad3cf6be504e35aaec10b7b9f3f34d` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:49.580452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b488572-e82a-4010-875c-dca978d0386a None admin] RBAC: Authorizing `identity:create_grant(project_id=debcfc4c17a34d8a80f13016124abfaf, user_id=8aec0c782060448fbf49b7cadf083a17, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:49.581947 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b488572-e82a-4010-875c-dca978d0386a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:49.585895 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5a870-8e88-44d2-984d-7197b15b0e2d None admin] RBAC: Authorizing `identity:create_grant(project_id=debcfc4c17a34d8a80f13016124abfaf, user_id=8aec0c782060448fbf49b7cadf083a17, role_id=c3ad3cf6be504e35aaec10b7b9f3f34d)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:49.587344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5a870-8e88-44d2-984d-7197b15b0e2d None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:49.648825 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 121/233] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:26:49 2022] PUT /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf/users/8aec0c782060448fbf49b7cadf083a17/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:26:49.673455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 113/234] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:26:49 2022] PUT /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf/users/8aec0c782060448fbf49b7cadf083a17/roles/c3ad3cf6be504e35aaec10b7b9f3f34d => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:26:50.370815 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d0b50-edc5-4c83-b5fe-9e201e129b44 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:50.371353 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d0b50-edc5-4c83-b5fe-9e201e129b44 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:50.371436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d0b50-edc5-4c83-b5fe-9e201e129b44 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:50.372987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 122/235] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:50.378380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee2cd7-4cf1-4413-a4a0-d40b7a0e4114 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:50.378677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee2cd7-4cf1-4413-a4a0-d40b7a0e4114 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:50.378961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee2cd7-4cf1-4413-a4a0-d40b7a0e4114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:50.405625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-a0ee2cd7-4cf1-4413-a4a0-d40b7a0e4114 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:50.406846 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-a0ee2cd7-4cf1-4413-a4a0-d40b7a0e4114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:50.495482 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 114/236] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:50.518679 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d0b50-edc5-4c83-b5fe-9e201e129b44 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:50.545235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7ddf9-66fb-4fdb-910d-ee5eaea4633a None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:50.545509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7ddf9-66fb-4fdb-910d-ee5eaea4633a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:50.545734 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7ddf9-66fb-4fdb-910d-ee5eaea4633a None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:50.547125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 115/237] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:50.552253 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966cce9-1e8b-47ba-877c-e32000719c0a None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:50.552547 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966cce9-1e8b-47ba-877c-e32000719c0a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:50.552859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966cce9-1e8b-47ba-877c-e32000719c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:50.573171 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-4966cce9-1e8b-47ba-877c-e32000719c0a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:50.574361 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-4966cce9-1e8b-47ba-877c-e32000719c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:50.686872 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 116/238] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:26:50.704779 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d0b50-edc5-4c83-b5fe-9e201e129b44 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:50.739407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966cce9-1e8b-47ba-877c-e32000719c0a None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:50.743012 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:50.761798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc16f92-f961-4d7e-9775-666b951d3e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:50.762613 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc16f92-f961-4d7e-9775-666b951d3e63 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:50.762896 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc16f92-f961-4d7e-9775-666b951d3e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:50.763173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc16f92-f961-4d7e-9775-666b951d3e63 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:50.784353 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc16f92-f961-4d7e-9775-666b951d3e63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:50.797026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-4bc16f92-f961-4d7e-9775-666b951d3e63 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:26:50.797026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:50.797026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:50.797026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:50.797026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:50.797026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:50.797026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:50.797026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:50.797026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:50.797026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:50.797026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:50.797026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:50.797026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:50.797026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:50.797026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:50.797026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:50.797916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:50.797916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:50.797916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:50.797916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:50.797916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:50.797916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:50.797916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:50.797916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:50.797916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:50.797916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:50.797916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:50.797916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:50.797916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:50.797916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:50.797916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:50.797916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:50.798705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:50.798705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:50.798705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:50.798705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:50.798705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:50.798705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:50.798705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:50.798705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:50.798705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:50.798705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:50.798705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:26:50.798705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:26:50.799307 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 123/239] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:50.810517 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc16f92-f961-4d7e-9775-666b951d3e63 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:50.832160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:50.936910 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966cce9-1e8b-47ba-877c-e32000719c0a None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:50.992933 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc16f92-f961-4d7e-9775-666b951d3e63 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:51.033301 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64416a-72b2-4c56-ab70-189201920051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:51.034115 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64416a-72b2-4c56-ab70-189201920051 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:51.034405 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64416a-72b2-4c56-ab70-189201920051 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:51.034687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64416a-72b2-4c56-ab70-189201920051 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:51.037343 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a64416a-72b2-4c56-ab70-189201920051 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:51.039171 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a64416a-72b2-4c56-ab70-189201920051 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:51.041884 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dbf3d-2151-44d3-aef0-e5947b0a1010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:51.042568 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dbf3d-2151-44d3-aef0-e5947b0a1010 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:51.042854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dbf3d-2151-44d3-aef0-e5947b0a1010 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:51.043137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dbf3d-2151-44d3-aef0-e5947b0a1010 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:51.044170 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61dbf3d-2151-44d3-aef0-e5947b0a1010 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:51.044488 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61dbf3d-2151-44d3-aef0-e5947b0a1010 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:51.045883 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61dbf3d-2151-44d3-aef0-e5947b0a1010 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:51.048199 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 117/240] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:51.054483 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64416a-72b2-4c56-ab70-189201920051 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:51.065624 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 124/241] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:26:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:51.070871 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dbf3d-2151-44d3-aef0-e5947b0a1010 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:51.131564 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:51.190663 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64416a-72b2-4c56-ab70-189201920051 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:51.205331 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dbf3d-2151-44d3-aef0-e5947b0a1010 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:51.209852 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:51.232382 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a55e82-7547-4eb1-ab03-da2096caaaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:51.233100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a55e82-7547-4eb1-ab03-da2096caaaac None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:51.233383 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a55e82-7547-4eb1-ab03-da2096caaaac None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:51.233657 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a55e82-7547-4eb1-ab03-da2096caaaac None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:51.234917 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a55e82-7547-4eb1-ab03-da2096caaaac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:51.237089 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a55e82-7547-4eb1-ab03-da2096caaaac None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:51.250480 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ef065-2684-4f3f-be8d-d2c566ff4b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:51.251470 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ef065-2684-4f3f-be8d-d2c566ff4b9c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:51.251835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ef065-2684-4f3f-be8d-d2c566ff4b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:51.252183 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ef065-2684-4f3f-be8d-d2c566ff4b9c None admin] PATH_INFO: `/v3/users/656a5744eaea44d7b045307b67a2d654` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:51.267667 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ef065-2684-4f3f-be8d-d2c566ff4b9c None admin] RBAC: Authorizing `identity:get_user(user_id=656a5744eaea44d7b045307b67a2d654)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:51.271986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ef065-2684-4f3f-be8d-d2c566ff4b9c None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:51.271986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 125/242] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:51 2022] GET /identity/v3/users/656a5744eaea44d7b045307b67a2d654 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:51.277634 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fe780-22d8-49f5-beed-2787d8c0b4e1 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:51.277941 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fe780-22d8-49f5-beed-2787d8c0b4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:51.278226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fe780-22d8-49f5-beed-2787d8c0b4e1 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:51.279811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 126/243] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:51.281773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fe780-22d8-49f5-beed-2787d8c0b4e1 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:51.325694 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 118/244] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 22:26:51 2022] POST /identity/v3/users => generated 293 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:26:51.327800 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f7c63-afa9-4252-8c8e-b8a3b4b04294 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:51.329110 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f7c63-afa9-4252-8c8e-b8a3b4b04294 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:51.329613 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f7c63-afa9-4252-8c8e-b8a3b4b04294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:51.354595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-fb1f7c63-afa9-4252-8c8e-b8a3b4b04294 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:51.357063 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-fb1f7c63-afa9-4252-8c8e-b8a3b4b04294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:51.407741 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fe780-22d8-49f5-beed-2787d8c0b4e1 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:51.418669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:51.450624 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23262317-8f02-4bb4-8058-31708dae12c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:51.451916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23262317-8f02-4bb4-8058-31708dae12c6 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:51.452512 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23262317-8f02-4bb4-8058-31708dae12c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:51.453193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23262317-8f02-4bb4-8058-31708dae12c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:51.454592 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 119/245] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:51.506731 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f7c63-afa9-4252-8c8e-b8a3b4b04294 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:51.550400 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23262317-8f02-4bb4-8058-31708dae12c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:51.553410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23262317-8f02-4bb4-8058-31708dae12c6 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:51.661483 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f7c63-afa9-4252-8c8e-b8a3b4b04294 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:51.664178 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:51.671385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 127/246] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:26:51.677240 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23262317-8f02-4bb4-8058-31708dae12c6 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:51.687119 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:51.699058 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019bae2-2a3f-4e57-80e1-e89c74b6722f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:51.700417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019bae2-2a3f-4e57-80e1-e89c74b6722f None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:51.700872 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019bae2-2a3f-4e57-80e1-e89c74b6722f None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:51.701315 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019bae2-2a3f-4e57-80e1-e89c74b6722f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:51.712362 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019bae2-2a3f-4e57-80e1-e89c74b6722f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:51.721189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-7019bae2-2a3f-4e57-80e1-e89c74b6722f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:51.721189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:51.721189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:51.721189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:51.721189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:51.721189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:51.721189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:51.721189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:51.721189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:51.721189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:51.721189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:51.721189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:51.721189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:51.721189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:51.721189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:51.721189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:51.729324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:51.729324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:51.729324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:51.729324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:51.729324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:51.729324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:51.729324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:51.729324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:51.729324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:51.729324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:51.729324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:51.729324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:51.729324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:51.729324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:51.729324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:51.729324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:51.730296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:51.730296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:51.730296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:51.730296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:51.730296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:51.730296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:51.730296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:51.730296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:51.730296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:51.730296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:51.730296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:51.730296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:26:51.730296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 120/247] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:26:51.732394 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019bae2-2a3f-4e57-80e1-e89c74b6722f None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:51.788917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23262317-8f02-4bb4-8058-31708dae12c6 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:51.837401 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfb695-943c-4622-b197-c389980aa30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:51.838885 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfb695-943c-4622-b197-c389980aa30f None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:51.839714 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfb695-943c-4622-b197-c389980aa30f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:51.840008 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfb695-943c-4622-b197-c389980aa30f None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:51.842593 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfb695-943c-4622-b197-c389980aa30f None admin] RBAC: Authorizing `identity:get_project(project_id=debcfc4c17a34d8a80f13016124abfaf)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:51.845675 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfb695-943c-4622-b197-c389980aa30f None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:51.847737 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 128/248] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:51 2022] GET /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:51.854647 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfb695-943c-4622-b197-c389980aa30f None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:51.890576 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019bae2-2a3f-4e57-80e1-e89c74b6722f None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:51.940036 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380edabe-683f-4c6a-b49f-5389f290fec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:51.940764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380edabe-683f-4c6a-b49f-5389f290fec7 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:51.941058 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380edabe-683f-4c6a-b49f-5389f290fec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:51.941340 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380edabe-683f-4c6a-b49f-5389f290fec7 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:51.942106 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380edabe-683f-4c6a-b49f-5389f290fec7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:51.942421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380edabe-683f-4c6a-b49f-5389f290fec7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:51.943830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380edabe-683f-4c6a-b49f-5389f290fec7 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:51.954621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 121/249] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:26:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:51.956436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:51.960801 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380edabe-683f-4c6a-b49f-5389f290fec7 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:51.988105 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfb695-943c-4622-b197-c389980aa30f None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:51.999208 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:52.045973 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b2e7b3-3b0a-4180-b02f-50e2270888d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:52.046939 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2e7b3-3b0a-4180-b02f-50e2270888d2 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:52.047719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2e7b3-3b0a-4180-b02f-50e2270888d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:52.048435 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2e7b3-3b0a-4180-b02f-50e2270888d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:52.049649 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b2e7b3-3b0a-4180-b02f-50e2270888d2 None admin] RBAC: Adding query filter params (role.id=917b4049bffd4db282dfe80ac37538be, scope.project.id=debcfc4c17a34d8a80f13016124abfaf, user.id=656a5744eaea44d7b045307b67a2d654) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:52.050456 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b2e7b3-3b0a-4180-b02f-50e2270888d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:52.052566 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b2e7b3-3b0a-4180-b02f-50e2270888d2 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:52.060918 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 129/250] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:51 2022] GET /identity/v3/role_assignments?user.id=656a5744eaea44d7b045307b67a2d654&scope.project.id=debcfc4c17a34d8a80f13016124abfaf&role.id=917b4049bffd4db282dfe80ac37538be => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:52.108115 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380edabe-683f-4c6a-b49f-5389f290fec7 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:52.123603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc640ac7-2f35-4005-a12b-ea18ea12842a None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:52.124138 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc640ac7-2f35-4005-a12b-ea18ea12842a None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:52.124615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc640ac7-2f35-4005-a12b-ea18ea12842a None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:52.126170 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 130/251] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:26:52.138290 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8d949-bef2-4311-949f-3d5ff81ca908 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:52.138598 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8d949-bef2-4311-949f-3d5ff81ca908 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:52.138851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8d949-bef2-4311-949f-3d5ff81ca908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:52.159874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57eb95-f047-4c79-ae47-a781cf863264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:52.160798 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57eb95-f047-4c79-ae47-a781cf863264 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:52.161084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57eb95-f047-4c79-ae47-a781cf863264 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:52.161363 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57eb95-f047-4c79-ae47-a781cf863264 None admin] PATH_INFO: `/v3/users/8ceda92aea5c4204a152f074a9e0d28d` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:52.162632 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57eb95-f047-4c79-ae47-a781cf863264 None admin] RBAC: Authorizing `identity:get_user(user_id=8ceda92aea5c4204a152f074a9e0d28d)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:52.164112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57eb95-f047-4c79-ae47-a781cf863264 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:52.166443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 122/252] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:51 2022] GET /identity/v3/users/8ceda92aea5c4204a152f074a9e0d28d => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:52.171564 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-3da8d949-bef2-4311-949f-3d5ff81ca908 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:52.172810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3da8d949-bef2-4311-949f-3d5ff81ca908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:52.180182 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57eb95-f047-4c79-ae47-a781cf863264 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:52.326572 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 131/253] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:52.345007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:52.355204 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57eb95-f047-4c79-ae47-a781cf863264 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:52.364656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8d949-bef2-4311-949f-3d5ff81ca908 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:52.420130 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32785b6e-93f1-4e88-8bb7-2c8bfa1a53c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:52.421036 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32785b6e-93f1-4e88-8bb7-2c8bfa1a53c0 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:52.421325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32785b6e-93f1-4e88-8bb7-2c8bfa1a53c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:52.421605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32785b6e-93f1-4e88-8bb7-2c8bfa1a53c0 None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:52.423616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32785b6e-93f1-4e88-8bb7-2c8bfa1a53c0 None admin] RBAC: Authorizing `identity:get_project(project_id=debcfc4c17a34d8a80f13016124abfaf)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:52.425112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32785b6e-93f1-4e88-8bb7-2c8bfa1a53c0 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:52.426874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 123/254] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:52 2022] GET /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:52.436605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32785b6e-93f1-4e88-8bb7-2c8bfa1a53c0 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:52.511420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:52.578321 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8d949-bef2-4311-949f-3d5ff81ca908 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:52.614539 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32785b6e-93f1-4e88-8bb7-2c8bfa1a53c0 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:52.645280 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7540a47-9e39-45be-8671-955e181d8d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:52.646078 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7540a47-9e39-45be-8671-955e181d8d6d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:52.646536 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7540a47-9e39-45be-8671-955e181d8d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:52.646956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7540a47-9e39-45be-8671-955e181d8d6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:52.650833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91275f0-9a28-48cd-9a77-09d9be995252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:52.651904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91275f0-9a28-48cd-9a77-09d9be995252 None admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:52.652362 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91275f0-9a28-48cd-9a77-09d9be995252 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:52.652827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91275f0-9a28-48cd-9a77-09d9be995252 None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf/users/8ceda92aea5c4204a152f074a9e0d28d/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:52.655022 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91275f0-9a28-48cd-9a77-09d9be995252 None admin] RBAC: Authorizing `identity:create_grant(project_id=debcfc4c17a34d8a80f13016124abfaf, user_id=8ceda92aea5c4204a152f074a9e0d28d, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:52.656793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91275f0-9a28-48cd-9a77-09d9be995252 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:52.668053 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7540a47-9e39-45be-8671-955e181d8d6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:52.676821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-e7540a47-9e39-45be-8671-955e181d8d6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:52.676821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:52.676821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:52.676821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:52.676821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:52.676821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:52.676821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:52.676821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:52.676821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:52.676821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:52.676821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:52.676821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:52.676821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:52.676821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:52.676821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:52.676821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:52.677895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:52.677895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:52.677895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:52.677895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:52.677895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:52.677895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:52.677895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:52.677895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:52.677895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:52.677895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:52.677895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:52.677895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:52.677895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:52.677895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:52.677895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:52.677895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:52.678903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:52.678903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:52.678903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:52.678903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:52.678903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:52.678903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:52.678903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:52.678903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:52.678903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:52.678903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:52.678903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:52.678903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:26:52.681948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 132/255] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:26:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:26:52.683493 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbc4c4-51ce-4fa3-b7f8-3721b92cf7d3 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:52.683936 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbc4c4-51ce-4fa3-b7f8-3721b92cf7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:52.684385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbc4c4-51ce-4fa3-b7f8-3721b92cf7d3 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:52.685924 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 133/256] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:52.690719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcbc4c4-51ce-4fa3-b7f8-3721b92cf7d3 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:52.734545 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 124/257] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:26:52 2022] PUT /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf/users/8ceda92aea5c4204a152f074a9e0d28d/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:26:52.736388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3351b-4037-40bc-8f46-dfdd444f6689 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:52.736883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3351b-4037-40bc-8f46-dfdd444f6689 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:52.737344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3351b-4037-40bc-8f46-dfdd444f6689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:52.770946 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-94a3351b-4037-40bc-8f46-dfdd444f6689 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:52.772635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-94a3351b-4037-40bc-8f46-dfdd444f6689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:52.813026 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:52.859642 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcbc4c4-51ce-4fa3-b7f8-3721b92cf7d3 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:52.917660 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a7825-133e-456b-9552-5342d661dc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:52.918507 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a7825-133e-456b-9552-5342d661dc13 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:52.919057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a7825-133e-456b-9552-5342d661dc13 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:52.919345 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a7825-133e-456b-9552-5342d661dc13 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:52.920439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a7825-133e-456b-9552-5342d661dc13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:52.920789 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a7825-133e-456b-9552-5342d661dc13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:52.922857 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a7825-133e-456b-9552-5342d661dc13 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:52.926192 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:52.933540 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 134/258] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:52.934733 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 125/259] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:52.939083 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a7825-133e-456b-9552-5342d661dc13 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:52.987589 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3351b-4037-40bc-8f46-dfdd444f6689 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:53.115589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a7825-133e-456b-9552-5342d661dc13 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:53.119358 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3351b-4037-40bc-8f46-dfdd444f6689 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:53.161734 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff791fe-29e0-490b-abac-aa2e606d310b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:53.162561 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff791fe-29e0-490b-abac-aa2e606d310b None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:53.162850 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff791fe-29e0-490b-abac-aa2e606d310b None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:53.163133 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff791fe-29e0-490b-abac-aa2e606d310b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:53.174315 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff791fe-29e0-490b-abac-aa2e606d310b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:53.184165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-5ff791fe-29e0-490b-abac-aa2e606d310b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:26:53.184165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:53.184165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:53.184165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:53.184165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:53.184165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:53.184165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:53.184165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:53.184165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:53.184165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:53.184165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:53.184165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:53.184165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:53.184165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:53.184165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:53.184165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:53.185074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:53.185074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:53.185074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:53.185074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:53.185074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:53.185074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:53.185074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:53.185074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:53.185074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:53.185074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:53.185074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:53.185074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:53.185074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:53.185074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:53.185074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:53.185074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:53.185866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:53.185866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:53.185866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:53.185866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:53.185866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:53.185866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:53.185866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:53.185866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:53.185866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:53.185866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:53.185866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:26:53.185866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:26:53.185866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 126/260] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:26:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:26:53.191832 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff791fe-29e0-490b-abac-aa2e606d310b None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:53.201881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107f8a9-28e4-4247-9e76-ad660b8c14a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:53.202772 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107f8a9-28e4-4247-9e76-ad660b8c14a3 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:53.203051 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107f8a9-28e4-4247-9e76-ad660b8c14a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:53.203325 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107f8a9-28e4-4247-9e76-ad660b8c14a3 None admin] PATH_INFO: `/v3/users/8aec0c782060448fbf49b7cadf083a17` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:53.205385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107f8a9-28e4-4247-9e76-ad660b8c14a3 None admin] RBAC: Authorizing `identity:get_user(user_id=8aec0c782060448fbf49b7cadf083a17)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:53.206875 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107f8a9-28e4-4247-9e76-ad660b8c14a3 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:53.208566 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 135/261] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:52 2022] GET /identity/v3/users/8aec0c782060448fbf49b7cadf083a17 => generated 283 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:53.208867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:53.214628 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107f8a9-28e4-4247-9e76-ad660b8c14a3 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:53.302297 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff791fe-29e0-490b-abac-aa2e606d310b None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:53.340628 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f5dcf-b0d8-4143-a05e-1a7e59941e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:53.341304 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:53.341665 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f5dcf-b0d8-4143-a05e-1a7e59941e44 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:53.341963 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f5dcf-b0d8-4143-a05e-1a7e59941e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:53.342250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f5dcf-b0d8-4143-a05e-1a7e59941e44 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:53.343009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f5dcf-b0d8-4143-a05e-1a7e59941e44 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:53.343323 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f5dcf-b0d8-4143-a05e-1a7e59941e44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:53.344774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f5dcf-b0d8-4143-a05e-1a7e59941e44 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:53.354489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 127/262] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:26:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:53.360214 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f5dcf-b0d8-4143-a05e-1a7e59941e44 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:53.455028 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107f8a9-28e4-4247-9e76-ad660b8c14a3 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:53.469722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:53.496344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f5dcf-b0d8-4143-a05e-1a7e59941e44 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:53.529499 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c3bc9-48cd-43de-803a-487f26214fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:53.530279 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c3bc9-48cd-43de-803a-487f26214fbb None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:53.530570 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c3bc9-48cd-43de-803a-487f26214fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:53.530915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c3bc9-48cd-43de-803a-487f26214fbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:53.532122 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313f3c5-bf52-4ebe-88a0-f1788a5a54e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:53.533142 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313f3c5-bf52-4ebe-88a0-f1788a5a54e6 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:53.533432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313f3c5-bf52-4ebe-88a0-f1788a5a54e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:53.533726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313f3c5-bf52-4ebe-88a0-f1788a5a54e6 None admin] PATH_INFO: `/v3/users/8aec0c782060448fbf49b7cadf083a17` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:53.534963 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9313f3c5-bf52-4ebe-88a0-f1788a5a54e6 None admin] RBAC: Authorizing `identity:get_user(user_id=8aec0c782060448fbf49b7cadf083a17)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:53.536441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9313f3c5-bf52-4ebe-88a0-f1788a5a54e6 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:53.538487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 128/263] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:53 2022] GET /identity/v3/users/8aec0c782060448fbf49b7cadf083a17 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:53.551560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313f3c5-bf52-4ebe-88a0-f1788a5a54e6 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:53.648830 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c3bc9-48cd-43de-803a-487f26214fbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:53.654279 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c3bc9-48cd-43de-803a-487f26214fbb None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:53.670297 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313f3c5-bf52-4ebe-88a0-f1788a5a54e6 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:53.711202 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c268b-a847-4fb5-8090-83588722c850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:53.711794 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c268b-a847-4fb5-8090-83588722c850 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:53.712083 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c268b-a847-4fb5-8090-83588722c850 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:53.712367 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c268b-a847-4fb5-8090-83588722c850 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:53.741641 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:53.760936 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:53.821267 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c268b-a847-4fb5-8090-83588722c850 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:53.823428 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c268b-a847-4fb5-8090-83588722c850 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:53.847812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 136/264] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 22:26:53.852997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c3bc9-48cd-43de-803a-487f26214fbb None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:53.949188 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 129/265] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:26:53.954734 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c268b-a847-4fb5-8090-83588722c850 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:54.004942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:54.091410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c268b-a847-4fb5-8090-83588722c850 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:54.139935 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf37f7a-ff30-4c01-ae53-0eb471ff9097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:54.140904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf37f7a-ff30-4c01-ae53-0eb471ff9097 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:54.141203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf37f7a-ff30-4c01-ae53-0eb471ff9097 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:54.141493 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf37f7a-ff30-4c01-ae53-0eb471ff9097 None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:54.145304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf37f7a-ff30-4c01-ae53-0eb471ff9097 None admin] RBAC: Authorizing `identity:get_project(project_id=debcfc4c17a34d8a80f13016124abfaf)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:54.146814 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf37f7a-ff30-4c01-ae53-0eb471ff9097 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:54.148872 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 130/266] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:53 2022] GET /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:54.154671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf37f7a-ff30-4c01-ae53-0eb471ff9097 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:54.183139 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c3bc9-48cd-43de-803a-487f26214fbb None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:54.308847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217b839-2509-4ef0-b61f-1b23776e020f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:54.315838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217b839-2509-4ef0-b61f-1b23776e020f None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:54.315838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217b839-2509-4ef0-b61f-1b23776e020f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:54.315838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217b839-2509-4ef0-b61f-1b23776e020f None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:54.315838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217b839-2509-4ef0-b61f-1b23776e020f None admin] RBAC: Authorizing `identity:get_project(project_id=debcfc4c17a34d8a80f13016124abfaf)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:54.315838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217b839-2509-4ef0-b61f-1b23776e020f None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:54.315838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 137/267] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:53 2022] GET /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf => generated 302 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:54.326994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217b839-2509-4ef0-b61f-1b23776e020f None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:54.358937 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf37f7a-ff30-4c01-ae53-0eb471ff9097 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:54.388948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:54.393520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:54.434746 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b37cf3-98e2-4908-ab30-6c0faf5e7a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:54.435427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b37cf3-98e2-4908-ab30-6c0faf5e7a0f None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:54.435771 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b37cf3-98e2-4908-ab30-6c0faf5e7a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:54.436055 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b37cf3-98e2-4908-ab30-6c0faf5e7a0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:54.436983 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b37cf3-98e2-4908-ab30-6c0faf5e7a0f None admin] RBAC: Adding query filter params (role.id=c3ad3cf6be504e35aaec10b7b9f3f34d, scope.project.id=debcfc4c17a34d8a80f13016124abfaf, user.id=8aec0c782060448fbf49b7cadf083a17) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:54.437302 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b37cf3-98e2-4908-ab30-6c0faf5e7a0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:54.438780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b37cf3-98e2-4908-ab30-6c0faf5e7a0f None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:54.454168 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 131/268] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:54 2022] GET /identity/v3/role_assignments?user.id=8aec0c782060448fbf49b7cadf083a17&scope.project.id=debcfc4c17a34d8a80f13016124abfaf&role.id=c3ad3cf6be504e35aaec10b7b9f3f34d => generated 608 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:54.538811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217b839-2509-4ef0-b61f-1b23776e020f None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:54.570533 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b73cc4-f07d-4ea1-b5e0-9f87a194d480 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:54.571164 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b73cc4-f07d-4ea1-b5e0-9f87a194d480 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:54.571665 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b73cc4-f07d-4ea1-b5e0-9f87a194d480 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:54.574459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 132/269] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:54.580354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8e897-c802-4e45-a657-7abd203c2d20 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:54.581053 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8e897-c802-4e45-a657-7abd203c2d20 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:54.581823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8e897-c802-4e45-a657-7abd203c2d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:54.590654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee631a-5f19-47b8-a209-b459c22cebaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:54.591760 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee631a-5f19-47b8-a209-b459c22cebaa None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:54.593173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee631a-5f19-47b8-a209-b459c22cebaa None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:54.593652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee631a-5f19-47b8-a209-b459c22cebaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:54.599814 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee631a-5f19-47b8-a209-b459c22cebaa None admin] RBAC: Adding query filter params (role.id=6ff7708b59984b9287cc81d46a57b4b4, scope.project.id=debcfc4c17a34d8a80f13016124abfaf, user.id=8aec0c782060448fbf49b7cadf083a17) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:54.600490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee631a-5f19-47b8-a209-b459c22cebaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:54.602208 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee631a-5f19-47b8-a209-b459c22cebaa None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:54.613952 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 138/270] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:54 2022] GET /identity/v3/role_assignments?user.id=8aec0c782060448fbf49b7cadf083a17&scope.project.id=debcfc4c17a34d8a80f13016124abfaf&role.id=6ff7708b59984b9287cc81d46a57b4b4 => generated 608 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:54.615314 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-03c8e897-c802-4e45-a657-7abd203c2d20 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:54.616853 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-03c8e897-c802-4e45-a657-7abd203c2d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:54.693751 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 133/271] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:54.718578 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee631a-5f19-47b8-a209-b459c22cebaa None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:54.831941 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:54.842342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee631a-5f19-47b8-a209-b459c22cebaa None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:54.886763 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1638b-8912-414d-8245-d36387831145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:54.887580 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1638b-8912-414d-8245-d36387831145 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:54.887865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1638b-8912-414d-8245-d36387831145 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:54.888141 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1638b-8912-414d-8245-d36387831145 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:54.898784 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1638b-8912-414d-8245-d36387831145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:54.906685 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92483897-6c9f-43fc-8978-a8b60119b8aa None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:54.907182 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92483897-6c9f-43fc-8978-a8b60119b8aa None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:54.907257 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92483897-6c9f-43fc-8978-a8b60119b8aa None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:54.910351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-74b1638b-8912-414d-8245-d36387831145 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:54.910351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:54.910351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:54.910351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:54.910351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:54.910351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:54.910351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:54.910351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:54.910351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:54.910351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:54.910351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:54.910351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:54.910351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:54.910351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:54.910351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:54.910351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:54.911218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:54.911218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:54.911218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:54.911218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:54.911218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:54.911218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:54.911218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:54.911218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:54.911218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:54.911218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:54.911218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:54.911218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:54.911218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:54.911218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:54.911218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:54.911218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:54.911987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:54.911987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:54.911987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:54.911987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:54.911987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:54.911987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:54.911987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:54.911987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:54.911987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:54.911987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:54.911987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:54.911987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:26:54.913037 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 134/272] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:54.913857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7113cf1-c65a-4765-af72-7c4de6be9d11 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:54.914148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7113cf1-c65a-4765-af72-7c4de6be9d11 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:54.914424 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7113cf1-c65a-4765-af72-7c4de6be9d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:54.915119 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 139/273] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:54.923379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1638b-8912-414d-8245-d36387831145 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:54.939244 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-e7113cf1-c65a-4765-af72-7c4de6be9d11 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:54.939888 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e7113cf1-c65a-4765-af72-7c4de6be9d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:55.043190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 135/274] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:26:55.055136 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1638b-8912-414d-8245-d36387831145 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:55.079835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c428-f66d-4430-afab-0f8c1676ef0b None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:55.080178 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c428-f66d-4430-afab-0f8c1676ef0b None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:55.080404 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c428-f66d-4430-afab-0f8c1676ef0b None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:55.083854 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 136/275] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:26:55.085682 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2c428-f66d-4430-afab-0f8c1676ef0b None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:55.096791 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b48c3d-e11c-4ea7-8950-24e09bf907d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:55.097598 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b48c3d-e11c-4ea7-8950-24e09bf907d1 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:55.097885 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b48c3d-e11c-4ea7-8950-24e09bf907d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:55.098164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b48c3d-e11c-4ea7-8950-24e09bf907d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:55.098970 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b48c3d-e11c-4ea7-8950-24e09bf907d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:55.099360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b48c3d-e11c-4ea7-8950-24e09bf907d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:55.101790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b48c3d-e11c-4ea7-8950-24e09bf907d1 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:55.111764 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 140/276] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:26:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:55.113350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073ba4d-dd93-4493-9f4b-c56cc9aae7ee None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:55.113861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073ba4d-dd93-4493-9f4b-c56cc9aae7ee None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:55.114308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073ba4d-dd93-4493-9f4b-c56cc9aae7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:55.136321 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-9073ba4d-dd93-4493-9f4b-c56cc9aae7ee None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:55.138816 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-9073ba4d-dd93-4493-9f4b-c56cc9aae7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:55.140494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:55.145527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:55.208342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 141/277] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:55.209215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073ba4d-dd93-4493-9f4b-c56cc9aae7ee None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:55.218483 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2c428-f66d-4430-afab-0f8c1676ef0b None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:55.259994 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ef8c0-2f46-44a7-8b28-6781f28bbdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:55.260955 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ef8c0-2f46-44a7-8b28-6781f28bbdf7 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:55.261133 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ef8c0-2f46-44a7-8b28-6781f28bbdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:55.261379 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ef8c0-2f46-44a7-8b28-6781f28bbdf7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:55.262543 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ef8c0-2f46-44a7-8b28-6781f28bbdf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:55.264000 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ef8c0-2f46-44a7-8b28-6781f28bbdf7 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:55.265810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 137/278] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:55.266397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ef8c0-2f46-44a7-8b28-6781f28bbdf7 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:55.325709 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073ba4d-dd93-4493-9f4b-c56cc9aae7ee None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:55.366837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a611e4-bc33-4504-a8e8-5f2ffd78c4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:55.366837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a611e4-bc33-4504-a8e8-5f2ffd78c4ad None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:55.366837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a611e4-bc33-4504-a8e8-5f2ffd78c4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:55.366837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a611e4-bc33-4504-a8e8-5f2ffd78c4ad None admin] PATH_INFO: `/v3/users/fc95004e6fe54619ab7cd139088cc591` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:55.376315 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:55.377793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ef8c0-2f46-44a7-8b28-6781f28bbdf7 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:55.378897 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a611e4-bc33-4504-a8e8-5f2ffd78c4ad None admin] RBAC: Authorizing `identity:get_user(user_id=fc95004e6fe54619ab7cd139088cc591)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:55.379527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:55.380358 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a611e4-bc33-4504-a8e8-5f2ffd78c4ad None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:55.382088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 142/279] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:55 2022] GET /identity/v3/users/fc95004e6fe54619ab7cd139088cc591 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:55.383030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a611e4-bc33-4504-a8e8-5f2ffd78c4ad None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:55.419495 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7998ec-c44d-49c1-998e-98b2cdeb4cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:55.421095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7998ec-c44d-49c1-998e-98b2cdeb4cef None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:55.421386 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7998ec-c44d-49c1-998e-98b2cdeb4cef None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:55.421666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7998ec-c44d-49c1-998e-98b2cdeb4cef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:55.433005 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7998ec-c44d-49c1-998e-98b2cdeb4cef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:55.440997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-7a7998ec-c44d-49c1-998e-98b2cdeb4cef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:26:55.440997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:55.440997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:55.440997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:55.440997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:55.440997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:55.440997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:55.440997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:55.440997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:55.440997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:55.440997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:55.440997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:55.440997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:55.440997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:55.440997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:55.440997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:55.441950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:55.441950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:55.441950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:55.441950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:55.441950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:55.441950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:55.441950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:55.441950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:55.441950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:55.441950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:55.441950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:55.441950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:55.441950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:55.441950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:55.441950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:55.441950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:55.442782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:55.442782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:55.442782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:55.442782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:55.442782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:55.442782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:55.442782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:55.442782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:55.442782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:55.442782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:55.442782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:26:55.442782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:26:55.445688 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 138/280] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:55.446886 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7998ec-c44d-49c1-998e-98b2cdeb4cef None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:55.493934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a611e4-bc33-4504-a8e8-5f2ffd78c4ad None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:55.529248 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9fbafa-371b-45f1-b96c-e2e0c06e281d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:55.529918 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fbafa-371b-45f1-b96c-e2e0c06e281d None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:55.530206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fbafa-371b-45f1-b96c-e2e0c06e281d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:55.530484 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fbafa-371b-45f1-b96c-e2e0c06e281d None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:55.531198 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9fbafa-371b-45f1-b96c-e2e0c06e281d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:55.532666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9fbafa-371b-45f1-b96c-e2e0c06e281d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:55.551307 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7998ec-c44d-49c1-998e-98b2cdeb4cef None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:55.584927 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975b753-6b16-4fae-8da8-fee55b126615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:55.585578 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975b753-6b16-4fae-8da8-fee55b126615 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:55.585865 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975b753-6b16-4fae-8da8-fee55b126615 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:55.586139 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975b753-6b16-4fae-8da8-fee55b126615 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:55.586964 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6975b753-6b16-4fae-8da8-fee55b126615 None admin] RBAC: Adding query filter params (role.id=1305747c745d49dc9a9718b7aa8aa4a9, user.id=fc95004e6fe54619ab7cd139088cc591, scope.system=all) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:55.587280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6975b753-6b16-4fae-8da8-fee55b126615 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:55.588767 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6975b753-6b16-4fae-8da8-fee55b126615 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:55.593788 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 139/281] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Jun 23 22:26:55 2022] GET /identity/v3/role_assignments?user.id=fc95004e6fe54619ab7cd139088cc591&scope.system=all&role.id=1305747c745d49dc9a9718b7aa8aa4a9 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:55.604870 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975b753-6b16-4fae-8da8-fee55b126615 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:55.630030 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:55.634418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 143/282] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 22:26:55 2022] POST /identity/v3/users => generated 293 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:26:55.731698 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975b753-6b16-4fae-8da8-fee55b126615 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:55.774486 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d3fbf-a29e-4814-aee7-e280a6a7eea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:55.775608 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d3fbf-a29e-4814-aee7-e280a6a7eea7 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:55.775959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d3fbf-a29e-4814-aee7-e280a6a7eea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:55.776245 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d3fbf-a29e-4814-aee7-e280a6a7eea7 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:55.777123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d3fbf-a29e-4814-aee7-e280a6a7eea7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:55.777438 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d3fbf-a29e-4814-aee7-e280a6a7eea7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:55.778947 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d3fbf-a29e-4814-aee7-e280a6a7eea7 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:55.789864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 140/283] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:26:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:55.796523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9fbafa-371b-45f1-b96c-e2e0c06e281d None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:55.860296 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:55.906465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9fbafa-371b-45f1-b96c-e2e0c06e281d None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:55.943329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83705175-a583-4a11-b1c2-7ce2f37e0131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:55.944210 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83705175-a583-4a11-b1c2-7ce2f37e0131 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:55.944494 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83705175-a583-4a11-b1c2-7ce2f37e0131 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:55.944803 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83705175-a583-4a11-b1c2-7ce2f37e0131 None admin] PATH_INFO: `/v3/users/656a5744eaea44d7b045307b67a2d654` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:55.946580 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83705175-a583-4a11-b1c2-7ce2f37e0131 None admin] RBAC: Authorizing `identity:get_user(user_id=656a5744eaea44d7b045307b67a2d654)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:55.948099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83705175-a583-4a11-b1c2-7ce2f37e0131 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:55.950200 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 144/284] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:55 2022] GET /identity/v3/users/656a5744eaea44d7b045307b67a2d654 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:55.956934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d3fbf-a29e-4814-aee7-e280a6a7eea7 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:55.973106 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16e28b-185d-4e5b-a7b7-c0202c22a4e9 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:55.973412 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16e28b-185d-4e5b-a7b7-c0202c22a4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:55.973687 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16e28b-185d-4e5b-a7b7-c0202c22a4e9 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:55.975396 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 145/285] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:55.981949 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ba0de-9749-47c5-8b60-78d5b9363245 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:55.982281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ba0de-9749-47c5-8b60-78d5b9363245 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:55.982567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ba0de-9749-47c5-8b60-78d5b9363245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:56.005415 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-3a1ba0de-9749-47c5-8b60-78d5b9363245 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:56.006603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3a1ba0de-9749-47c5-8b60-78d5b9363245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:56.075014 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:56.089149 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 146/286] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:56.090816 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d3fbf-a29e-4814-aee7-e280a6a7eea7 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:56.113535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ba0de-9749-47c5-8b60-78d5b9363245 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:56.129919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af60ff-8047-422e-a39f-266288f832a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:56.131030 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af60ff-8047-422e-a39f-266288f832a0 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:56.131460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af60ff-8047-422e-a39f-266288f832a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:56.131880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af60ff-8047-422e-a39f-266288f832a0 None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:56.133303 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af60ff-8047-422e-a39f-266288f832a0 None admin] RBAC: Authorizing `identity:get_project(project_id=debcfc4c17a34d8a80f13016124abfaf)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:56.135490 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af60ff-8047-422e-a39f-266288f832a0 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:56.137453 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 141/287] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:55 2022] GET /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:56.143384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af60ff-8047-422e-a39f-266288f832a0 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:56.176076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:56.225076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ba0de-9749-47c5-8b60-78d5b9363245 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:56.261123 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97cb05-1189-4318-a6ed-1921d9a709a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:56.261935 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97cb05-1189-4318-a6ed-1921d9a709a3 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:56.262226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97cb05-1189-4318-a6ed-1921d9a709a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:56.262508 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97cb05-1189-4318-a6ed-1921d9a709a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:56.276140 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f97cb05-1189-4318-a6ed-1921d9a709a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:56.281827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-4f97cb05-1189-4318-a6ed-1921d9a709a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:56.281827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:56.281827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:56.281827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:56.281827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:56.281827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:56.281827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:56.281827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:56.281827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:56.281827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:56.281827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:56.281827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:56.281827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:56.281827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:56.281827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:56.281827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:56.282690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:56.282690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:56.282690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:56.282690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:56.282690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:56.282690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:56.282690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:56.282690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:56.282690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:56.282690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:56.282690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:56.282690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:56.282690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:56.282690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:56.282690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:56.282690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:56.283519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:56.283519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:56.283519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:56.283519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:56.283519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:56.283519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:56.283519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:56.283519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:56.283519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:56.283519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:56.283519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:56.283519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:26:56.283519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 147/288] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:26:56.288033 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af60ff-8047-422e-a39f-266288f832a0 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:56.292664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97cb05-1189-4318-a6ed-1921d9a709a3 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:56.374924 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e8043-4acf-4240-aec3-40e51b028bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:56.376134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e8043-4acf-4240-aec3-40e51b028bda None admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:56.376742 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e8043-4acf-4240-aec3-40e51b028bda None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:56.377195 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e8043-4acf-4240-aec3-40e51b028bda None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf/users/656a5744eaea44d7b045307b67a2d654/roles/917b4049bffd4db282dfe80ac37538be` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:56.386194 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e8043-4acf-4240-aec3-40e51b028bda None admin] RBAC: Authorizing `identity:create_grant(project_id=debcfc4c17a34d8a80f13016124abfaf, user_id=656a5744eaea44d7b045307b67a2d654, role_id=917b4049bffd4db282dfe80ac37538be)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:56.388233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e8043-4acf-4240-aec3-40e51b028bda None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:56.396945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97cb05-1189-4318-a6ed-1921d9a709a3 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:56.412396 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:56.434945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812925c5-4d07-4427-ad0e-8b8a684189d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:56.435539 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812925c5-4d07-4427-ad0e-8b8a684189d8 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:56.435892 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812925c5-4d07-4427-ad0e-8b8a684189d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:56.436183 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812925c5-4d07-4427-ad0e-8b8a684189d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:56.436975 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812925c5-4d07-4427-ad0e-8b8a684189d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:56.437293 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812925c5-4d07-4427-ad0e-8b8a684189d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:56.438730 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812925c5-4d07-4427-ad0e-8b8a684189d8 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:56.448885 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 148/289] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:26:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:56.454450 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812925c5-4d07-4427-ad0e-8b8a684189d8 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:56.487959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 142/290] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:26:56 2022] PUT /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf/users/656a5744eaea44d7b045307b67a2d654/roles/917b4049bffd4db282dfe80ac37538be => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:26:56.589174 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812925c5-4d07-4427-ad0e-8b8a684189d8 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:56.636498 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf9a0d-0001-4d88-84d3-35f2dd0bbf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:56.637481 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf9a0d-0001-4d88-84d3-35f2dd0bbf46 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:56.637773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf9a0d-0001-4d88-84d3-35f2dd0bbf46 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:56.638051 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf9a0d-0001-4d88-84d3-35f2dd0bbf46 None admin] PATH_INFO: `/v3/users/8ceda92aea5c4204a152f074a9e0d28d` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:56.639329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf9a0d-0001-4d88-84d3-35f2dd0bbf46 None admin] RBAC: Authorizing `identity:get_user(user_id=8ceda92aea5c4204a152f074a9e0d28d)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:56.641991 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf9a0d-0001-4d88-84d3-35f2dd0bbf46 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:56.642917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 149/291] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:56 2022] GET /identity/v3/users/8ceda92aea5c4204a152f074a9e0d28d => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:56.650296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e8043-4acf-4240-aec3-40e51b028bda None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:56.671009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:56.777315 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e8043-4acf-4240-aec3-40e51b028bda None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:56.823846 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d020c-aa26-4168-a4cd-a9d764bc7c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:56.824519 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d020c-aa26-4168-a4cd-a9d764bc7c21 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:56.824886 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d020c-aa26-4168-a4cd-a9d764bc7c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:56.825179 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d020c-aa26-4168-a4cd-a9d764bc7c21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:56.896206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d020c-aa26-4168-a4cd-a9d764bc7c21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:56.898092 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d020c-aa26-4168-a4cd-a9d764bc7c21 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:56.925122 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:57.046480 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 143/292] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:26:57.051954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf9a0d-0001-4d88-84d3-35f2dd0bbf46 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:57.128306 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:57.197874 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf9a0d-0001-4d88-84d3-35f2dd0bbf46 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:57.243331 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd32aa9-d22e-4429-b0b0-ab4d96e9c89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:57.244441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd32aa9-d22e-4429-b0b0-ab4d96e9c89c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:57.244766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd32aa9-d22e-4429-b0b0-ab4d96e9c89c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:57.245049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd32aa9-d22e-4429-b0b0-ab4d96e9c89c None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:57.249296 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd32aa9-d22e-4429-b0b0-ab4d96e9c89c None admin] RBAC: Authorizing `identity:get_project(project_id=debcfc4c17a34d8a80f13016124abfaf)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:57.250767 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd32aa9-d22e-4429-b0b0-ab4d96e9c89c None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:57.252718 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 150/293] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:26:57 2022] GET /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:57.258252 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d020c-aa26-4168-a4cd-a9d764bc7c21 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:57.423258 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d020c-aa26-4168-a4cd-a9d764bc7c21 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:57.444156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:57.468108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bedbebd-2a05-4371-a1d2-e1d64c764de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:57.468798 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedbebd-2a05-4371-a1d2-e1d64c764de3 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:57.469084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedbebd-2a05-4371-a1d2-e1d64c764de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:57.469365 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedbebd-2a05-4371-a1d2-e1d64c764de3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:57.470293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bedbebd-2a05-4371-a1d2-e1d64c764de3 None admin] RBAC: Adding query filter params (role.id=1305747c745d49dc9a9718b7aa8aa4a9, scope.project.id=debcfc4c17a34d8a80f13016124abfaf, user.id=8ceda92aea5c4204a152f074a9e0d28d) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:57.470825 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bedbebd-2a05-4371-a1d2-e1d64c764de3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:57.472556 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bedbebd-2a05-4371-a1d2-e1d64c764de3 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:57.479188 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 144/294] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:57 2022] GET /identity/v3/role_assignments?user.id=8ceda92aea5c4204a152f074a9e0d28d&scope.project.id=debcfc4c17a34d8a80f13016124abfaf&role.id=1305747c745d49dc9a9718b7aa8aa4a9 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:57.957262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da41b2-f45a-4622-a1dd-a888f70671c0 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:57.957775 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da41b2-f45a-4622-a1dd-a888f70671c0 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:57.958109 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da41b2-f45a-4622-a1dd-a888f70671c0 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:57.959586 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 151/295] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:57.967527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a75452-8bfd-4567-8bf5-5c5fd7cf9f27 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:57.967819 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a75452-8bfd-4567-8bf5-5c5fd7cf9f27 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:57.968100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a75452-8bfd-4567-8bf5-5c5fd7cf9f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:57.985246 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-c7a75452-8bfd-4567-8bf5-5c5fd7cf9f27 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:57.986428 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c7a75452-8bfd-4567-8bf5-5c5fd7cf9f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:58.064247 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 145/296] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:58.089759 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da41b2-f45a-4622-a1dd-a888f70671c0 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:58.197555 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da41b2-f45a-4622-a1dd-a888f70671c0 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:58.211256 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:58.232293 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050dbcb-1250-446e-b499-c159941b9648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:58.233125 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050dbcb-1250-446e-b499-c159941b9648 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:58.233407 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050dbcb-1250-446e-b499-c159941b9648 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:58.233682 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050dbcb-1250-446e-b499-c159941b9648 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:58.245564 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050dbcb-1250-446e-b499-c159941b9648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:58.253386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-2050dbcb-1250-446e-b499-c159941b9648 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:58.253386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:58.253386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:58.253386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:58.253386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:58.253386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:58.253386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:58.253386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:58.253386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:58.253386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:58.253386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:58.253386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:58.253386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:58.253386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:58.253386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:58.253386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:58.254252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:58.254252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:58.254252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:58.254252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:58.254252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:58.254252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:58.254252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:58.254252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:58.254252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:58.254252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:58.254252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:58.254252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:58.254252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:58.254252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:58.254252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:58.254252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:58.255473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:58.255473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:58.255473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:58.255473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:58.255473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:58.255473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:58.255473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:58.255473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:58.255473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:58.255473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:58.255473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:26:58.255473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:26:58.255473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 152/297] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:58.260925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a75452-8bfd-4567-8bf5-5c5fd7cf9f27 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:58.281979 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290d2c2-6d5a-4e3d-9821-abfbb2b225b7 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:58.282301 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290d2c2-6d5a-4e3d-9821-abfbb2b225b7 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:58.282583 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290d2c2-6d5a-4e3d-9821-abfbb2b225b7 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:58.284625 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 153/298] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:26:58.293792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51048089-65cf-474f-b235-b45549752d93 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:58.294081 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51048089-65cf-474f-b235-b45549752d93 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:58.294417 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51048089-65cf-474f-b235-b45549752d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:58.315565 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-51048089-65cf-474f-b235-b45549752d93 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:58.317183 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-51048089-65cf-474f-b235-b45549752d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:58.375454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a75452-8bfd-4567-8bf5-5c5fd7cf9f27 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:58.387661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:58.408228 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 154/299] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:26:58.415683 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d36d4-65c5-4043-a416-5e6c1fc79297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:58.416357 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d36d4-65c5-4043-a416-5e6c1fc79297 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:58.416670 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d36d4-65c5-4043-a416-5e6c1fc79297 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:58.416955 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d36d4-65c5-4043-a416-5e6c1fc79297 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:58.417702 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d36d4-65c5-4043-a416-5e6c1fc79297 None admin] RBAC: Adding query filter params (name=member) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:58.418009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d36d4-65c5-4043-a416-5e6c1fc79297 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:58.419647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d36d4-65c5-4043-a416-5e6c1fc79297 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:58.428171 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 146/300] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:26:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:58.428945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51048089-65cf-474f-b235-b45549752d93 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:58.434927 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d36d4-65c5-4043-a416-5e6c1fc79297 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:58.535756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d36d4-65c5-4043-a416-5e6c1fc79297 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:58.541042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51048089-65cf-474f-b235-b45549752d93 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:58.561016 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:58.572936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b63d14-d72c-47f6-8e3a-40e769445513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:58.573814 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b63d14-d72c-47f6-8e3a-40e769445513 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:58.574202 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b63d14-d72c-47f6-8e3a-40e769445513 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:58.574375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b63d14-d72c-47f6-8e3a-40e769445513 None admin] PATH_INFO: `/v3/users/fc95004e6fe54619ab7cd139088cc591` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:58.575585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b63d14-d72c-47f6-8e3a-40e769445513 None admin] RBAC: Authorizing `identity:get_user(user_id=fc95004e6fe54619ab7cd139088cc591)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:58.577079 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b63d14-d72c-47f6-8e3a-40e769445513 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:58.578897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 147/301] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:58 2022] GET /identity/v3/users/fc95004e6fe54619ab7cd139088cc591 => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:58.581476 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20958e-24fa-4f0a-8c03-70e16923d65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:58.582516 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20958e-24fa-4f0a-8c03-70e16923d65a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:58.583446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20958e-24fa-4f0a-8c03-70e16923d65a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:58.583964 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20958e-24fa-4f0a-8c03-70e16923d65a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:58.584133 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b63d14-d72c-47f6-8e3a-40e769445513 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:58.596841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20958e-24fa-4f0a-8c03-70e16923d65a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:58.605689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-df20958e-24fa-4f0a-8c03-70e16923d65a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:26:58.605689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:58.605689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:58.605689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:58.605689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:58.605689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:58.605689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:58.605689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:58.605689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:58.605689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:58.605689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:58.605689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:58.605689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:58.605689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:58.605689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:58.605689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:58.607231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:58.607231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:58.607231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:58.607231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:58.607231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:58.607231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:58.607231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:58.607231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:58.607231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:58.607231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:58.607231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:58.607231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:58.607231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:58.607231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:58.607231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:58.607231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:58.608228 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:58.608228 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:58.608228 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:58.608228 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:58.608228 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:58.608228 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:58.608228 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:58.608228 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:58.608228 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:58.608228 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:58.608228 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:26:58.608228 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:26:58.610598 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 155/302] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:58.615129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a0626-4e1c-4487-89ea-063d9e8c8b94 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:58.615259 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a0626-4e1c-4487-89ea-063d9e8c8b94 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:58.615514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a0626-4e1c-4487-89ea-063d9e8c8b94 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:58.617626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 156/303] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:58.621413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a0626-4e1c-4487-89ea-063d9e8c8b94 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:58.632833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:58.697123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b63d14-d72c-47f6-8e3a-40e769445513 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:58.731377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21ef7e-825f-439a-b5c0-4386af298cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:58.732246 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21ef7e-825f-439a-b5c0-4386af298cfe None admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:58.732607 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21ef7e-825f-439a-b5c0-4386af298cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:58.732839 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21ef7e-825f-439a-b5c0-4386af298cfe None admin] PATH_INFO: `/v3/system/users/fc95004e6fe54619ab7cd139088cc591/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:58.734345 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21ef7e-825f-439a-b5c0-4386af298cfe None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=fc95004e6fe54619ab7cd139088cc591, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:58.735637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a0626-4e1c-4487-89ea-063d9e8c8b94 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:58.735742 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21ef7e-825f-439a-b5c0-4386af298cfe None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:58.771032 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e71dc0-8bf9-4deb-916a-b362f2aec2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:58.771702 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e71dc0-8bf9-4deb-916a-b362f2aec2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:58.771988 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e71dc0-8bf9-4deb-916a-b362f2aec2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:58.772266 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e71dc0-8bf9-4deb-916a-b362f2aec2f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:58.773047 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e71dc0-8bf9-4deb-916a-b362f2aec2f8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:58.773359 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e71dc0-8bf9-4deb-916a-b362f2aec2f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:58.774865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e71dc0-8bf9-4deb-916a-b362f2aec2f8 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:58.783705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 157/304] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:26:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:58.784752 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10857545-7d63-45e7-8952-6f67b5d6a53f None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:58.785160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10857545-7d63-45e7-8952-6f67b5d6a53f None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:58.785447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10857545-7d63-45e7-8952-6f67b5d6a53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:58.807539 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-10857545-7d63-45e7-8952-6f67b5d6a53f None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:58.807539 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-10857545-7d63-45e7-8952-6f67b5d6a53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:58.822187 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:58.826609 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 148/305] 10.222.0.33 () {58 vars in 1334 bytes} [Thu Jun 23 22:26:58 2022] PUT /identity/v3/system/users/fc95004e6fe54619ab7cd139088cc591/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:26:58.836724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21ef7e-825f-439a-b5c0-4386af298cfe None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:58.883465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 158/306] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:58.912820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10857545-7d63-45e7-8952-6f67b5d6a53f None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:58.965671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21ef7e-825f-439a-b5c0-4386af298cfe None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:58.969281 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:59.012341 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37bdb6e-524f-49a4-877d-c255f16829f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:59.013271 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bdb6e-524f-49a4-877d-c255f16829f1 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:59.013558 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bdb6e-524f-49a4-877d-c255f16829f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:59.013865 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bdb6e-524f-49a4-877d-c255f16829f1 None admin] PATH_INFO: `/v3/users/faf96ff3ffe54a2aa420bb5dba3cc2b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:59.028282 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37bdb6e-524f-49a4-877d-c255f16829f1 None admin] RBAC: Authorizing `identity:get_user(user_id=faf96ff3ffe54a2aa420bb5dba3cc2b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:59.029811 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37bdb6e-524f-49a4-877d-c255f16829f1 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:59.031711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 149/307] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:58 2022] GET /identity/v3/users/faf96ff3ffe54a2aa420bb5dba3cc2b4 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:59.038508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37bdb6e-524f-49a4-877d-c255f16829f1 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:59.051718 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10857545-7d63-45e7-8952-6f67b5d6a53f None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:59.095514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491cc47-370c-4a9f-ab0e-3c827b9e34ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:59.096808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491cc47-370c-4a9f-ab0e-3c827b9e34ec None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:59.097106 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491cc47-370c-4a9f-ab0e-3c827b9e34ec None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:59.097393 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491cc47-370c-4a9f-ab0e-3c827b9e34ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:59.098746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491cc47-370c-4a9f-ab0e-3c827b9e34ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:59.100450 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491cc47-370c-4a9f-ab0e-3c827b9e34ec None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:59.102616 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 159/308] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:59.104925 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db0d3f-4999-4a65-9209-414c375a8c50 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:59.104925 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db0d3f-4999-4a65-9209-414c375a8c50 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:59.105165 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db0d3f-4999-4a65-9209-414c375a8c50 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:59.106229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 160/309] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:59.109588 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db0d3f-4999-4a65-9209-414c375a8c50 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:59.133816 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:59.160506 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37bdb6e-524f-49a4-877d-c255f16829f1 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:59.198889 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca09df-c2b2-4fdb-8756-3a4ac554dc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:59.199563 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca09df-c2b2-4fdb-8756-3a4ac554dc36 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:59.199856 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca09df-c2b2-4fdb-8756-3a4ac554dc36 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:59.200162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca09df-c2b2-4fdb-8756-3a4ac554dc36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:59.201013 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca09df-c2b2-4fdb-8756-3a4ac554dc36 None admin] RBAC: Adding query filter params (role.id=917b4049bffd4db282dfe80ac37538be, user.id=faf96ff3ffe54a2aa420bb5dba3cc2b4, scope.system=all) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:59.201330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca09df-c2b2-4fdb-8756-3a4ac554dc36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:59.202784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca09df-c2b2-4fdb-8756-3a4ac554dc36 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:59.208248 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 150/310] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Jun 23 22:26:59 2022] GET /identity/v3/role_assignments?user.id=faf96ff3ffe54a2aa420bb5dba3cc2b4&scope.system=all&role.id=917b4049bffd4db282dfe80ac37538be => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:59.209237 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3710b16-12f5-43d9-972a-3129010bcd59 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:59.209532 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3710b16-12f5-43d9-972a-3129010bcd59 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:59.209815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3710b16-12f5-43d9-972a-3129010bcd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:59.230159 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db0d3f-4999-4a65-9209-414c375a8c50 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:59.231404 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-b3710b16-12f5-43d9-972a-3129010bcd59 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:59.232921 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-b3710b16-12f5-43d9-972a-3129010bcd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:59.286570 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848d646-0a06-4204-9a93-d10f81ce4123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:59.288304 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848d646-0a06-4204-9a93-d10f81ce4123 None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:59.288794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848d646-0a06-4204-9a93-d10f81ce4123 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:59.289683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848d646-0a06-4204-9a93-d10f81ce4123 None admin] PATH_INFO: `/v3/groups` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:59.291208 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848d646-0a06-4204-9a93-d10f81ce4123 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:59.293922 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848d646-0a06-4204-9a93-d10f81ce4123 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:59.305799 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:59.320613 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 151/311] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:26:59.345825 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3710b16-12f5-43d9-972a-3129010bcd59 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:59.381983 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 161/312] 10.222.0.33 () {60 vars in 1134 bytes} [Thu Jun 23 22:26:59 2022] POST /identity/v3/groups => generated 229 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:26:59.460295 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3710b16-12f5-43d9-972a-3129010bcd59 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:59.499571 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffc725-da5f-409d-880b-3f3885108b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:59.500403 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffc725-da5f-409d-880b-3f3885108b12 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:59.500752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffc725-da5f-409d-880b-3f3885108b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:59.501030 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffc725-da5f-409d-880b-3f3885108b12 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:59.513808 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffc725-da5f-409d-880b-3f3885108b12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:59.522019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-50ffc725-da5f-409d-880b-3f3885108b12 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:26:59.522019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:59.522019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:59.522019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:59.522019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:59.522019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:59.522019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:59.522019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:59.522019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:59.522019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:59.522019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:59.522019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:59.522019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:59.522019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:59.522019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:59.522019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:59.522995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:59.522995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:59.522995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:59.522995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:59.522995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:59.522995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:59.522995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:59.522995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:59.522995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:59.522995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:59.522995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:59.522995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:59.522995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:59.522995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:59.522995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:59.522995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:59.523793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:59.523793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:59.523793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:59.523793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:59.523793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:59.523793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:59.523793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:59.523793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:59.523793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:59.523793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:59.523793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:26:59.523793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:26:59.523793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 152/313] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:59.531193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848d646-0a06-4204-9a93-d10f81ce4123 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:59.631221 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:59.652353 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848d646-0a06-4204-9a93-d10f81ce4123 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:59.692302 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f1925-896c-43d6-96b2-b865f151a9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:59.693368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f1925-896c-43d6-96b2-b865f151a9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:59.693938 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f1925-896c-43d6-96b2-b865f151a9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:59.694432 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f1925-896c-43d6-96b2-b865f151a9d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:59.695459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f1925-896c-43d6-96b2-b865f151a9d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:59.695972 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f1925-896c-43d6-96b2-b865f151a9d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:59.698559 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f1925-896c-43d6-96b2-b865f151a9d0 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:59.708131 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 162/314] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:26:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:59.714330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffc725-da5f-409d-880b-3f3885108b12 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:59.734582 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:59.824428 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffc725-da5f-409d-880b-3f3885108b12 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:59.859418 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3a22a-af13-4c95-8d0f-661f595c4566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:59.860676 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3a22a-af13-4c95-8d0f-661f595c4566 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:59.861218 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3a22a-af13-4c95-8d0f-661f595c4566 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:59.861687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3a22a-af13-4c95-8d0f-661f595c4566 None admin] PATH_INFO: `/v3/users/656a5744eaea44d7b045307b67a2d654` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:59.863092 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3a22a-af13-4c95-8d0f-661f595c4566 None admin] RBAC: Authorizing `identity:get_user(user_id=656a5744eaea44d7b045307b67a2d654)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:59.864782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3a22a-af13-4c95-8d0f-661f595c4566 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:59.866959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 153/315] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:26:59 2022] GET /identity/v3/users/656a5744eaea44d7b045307b67a2d654 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:59.876630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f1925-896c-43d6-96b2-b865f151a9d0 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:59.985914 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f1925-896c-43d6-96b2-b865f151a9d0 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:00.029735 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93f20f-bef1-42a3-95fc-cba3b061a7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:00.030405 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93f20f-bef1-42a3-95fc-cba3b061a7ed None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:00.030696 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93f20f-bef1-42a3-95fc-cba3b061a7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:00.030978 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93f20f-bef1-42a3-95fc-cba3b061a7ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:00.047542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:00.110872 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93f20f-bef1-42a3-95fc-cba3b061a7ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:00.112355 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93f20f-bef1-42a3-95fc-cba3b061a7ed None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:00.219956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 163/316] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:00.225854 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3a22a-af13-4c95-8d0f-661f595c4566 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:00.320084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:00.336909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3a22a-af13-4c95-8d0f-661f595c4566 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:00.374070 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4a52a-f959-4c24-8a76-2a822c49ed88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:00.375630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4a52a-f959-4c24-8a76-2a822c49ed88 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:00.375921 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4a52a-f959-4c24-8a76-2a822c49ed88 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:00.376203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4a52a-f959-4c24-8a76-2a822c49ed88 None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:00.377455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4a52a-f959-4c24-8a76-2a822c49ed88 None admin] RBAC: Authorizing `identity:get_project(project_id=debcfc4c17a34d8a80f13016124abfaf)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:00.379116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4a52a-f959-4c24-8a76-2a822c49ed88 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:00.381190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 154/317] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:27:00 2022] GET /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:00.387959 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93f20f-bef1-42a3-95fc-cba3b061a7ed None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:00.440317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:00.502130 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93f20f-bef1-42a3-95fc-cba3b061a7ed None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:00.541115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9f3eb-cdf1-4a17-b5dd-fc91e95a378d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:00.541712 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9f3eb-cdf1-4a17-b5dd-fc91e95a378d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:00.542004 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9f3eb-cdf1-4a17-b5dd-fc91e95a378d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:00.542288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9f3eb-cdf1-4a17-b5dd-fc91e95a378d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:00.543136 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9f3eb-cdf1-4a17-b5dd-fc91e95a378d None admin] RBAC: Adding query filter params (role.id=917b4049bffd4db282dfe80ac37538be, scope.project.id=debcfc4c17a34d8a80f13016124abfaf, user.id=656a5744eaea44d7b045307b67a2d654) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:00.543705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9f3eb-cdf1-4a17-b5dd-fc91e95a378d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:00.545191 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9f3eb-cdf1-4a17-b5dd-fc91e95a378d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:00.551726 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 164/318] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:27:00 2022] GET /identity/v3/role_assignments?user.id=656a5744eaea44d7b045307b67a2d654&scope.project.id=debcfc4c17a34d8a80f13016124abfaf&role.id=917b4049bffd4db282dfe80ac37538be => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:00.974675 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777e443-143f-4fed-b1b8-fdefcb3deec8 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:00.974887 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777e443-143f-4fed-b1b8-fdefcb3deec8 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:00.975174 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777e443-143f-4fed-b1b8-fdefcb3deec8 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:00.976704 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 155/319] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:00.982296 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6df52-4474-4085-92c8-9cedb499517a None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:00.982599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6df52-4474-4085-92c8-9cedb499517a None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:00.982885 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6df52-4474-4085-92c8-9cedb499517a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:00.999830 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-acf6df52-4474-4085-92c8-9cedb499517a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:01.001051 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-acf6df52-4474-4085-92c8-9cedb499517a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:01.070423 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 165/320] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:01.094136 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777e443-143f-4fed-b1b8-fdefcb3deec8 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:01.198741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777e443-143f-4fed-b1b8-fdefcb3deec8 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:01.233121 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a18ab-3dc9-4670-bc86-5be1b7bdda99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:01.233934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a18ab-3dc9-4670-bc86-5be1b7bdda99 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:01.234219 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a18ab-3dc9-4670-bc86-5be1b7bdda99 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:01.234498 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a18ab-3dc9-4670-bc86-5be1b7bdda99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:01.245483 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a18ab-3dc9-4670-bc86-5be1b7bdda99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:01.253489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-913a18ab-3dc9-4670-bc86-5be1b7bdda99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:27:01.253489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:01.253489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:01.253489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:01.253489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:01.253489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:01.253489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:01.253489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:01.253489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:01.253489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:01.253489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:01.253489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:01.253489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:01.253489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:01.253489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:01.253489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:01.254354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:01.254354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:01.254354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:01.254354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:01.254354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:01.254354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:01.254354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:01.254354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:01.254354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:01.254354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:01.254354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:01.254354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:01.254354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:01.254354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:01.254354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:01.254354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:01.255135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:01.255135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:01.255135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:01.255135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:01.255135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:01.255135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:01.255135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:01.255135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:01.255135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:01.255135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:01.255135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:27:01.255135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:01.256320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 156/321] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:01.261036 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6df52-4474-4085-92c8-9cedb499517a None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:01.307275 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:01.369040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6df52-4474-4085-92c8-9cedb499517a None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:01.385465 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016534dd-292f-4d59-b6bf-98863a9a02a8 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:01.385776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016534dd-292f-4d59-b6bf-98863a9a02a8 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:01.386026 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016534dd-292f-4d59-b6bf-98863a9a02a8 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:01.387544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 157/322] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:27:01.392360 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2654b1-8b6a-4a49-b93d-c4f4ce965118 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:01.392686 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2654b1-8b6a-4a49-b93d-c4f4ce965118 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:01.392972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2654b1-8b6a-4a49-b93d-c4f4ce965118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:01.409299 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-5d2654b1-8b6a-4a49-b93d-c4f4ce965118 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:01.410489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-5d2654b1-8b6a-4a49-b93d-c4f4ce965118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:01.418719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfe942-86dd-48e8-8df4-2de32a84f65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:01.418890 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:01.419396 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfe942-86dd-48e8-8df4-2de32a84f65d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:01.419686 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfe942-86dd-48e8-8df4-2de32a84f65d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:01.419966 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfe942-86dd-48e8-8df4-2de32a84f65d None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:01.420779 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dfe942-86dd-48e8-8df4-2de32a84f65d None admin] RBAC: Adding query filter params (name=member) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:01.421096 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dfe942-86dd-48e8-8df4-2de32a84f65d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:01.422487 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dfe942-86dd-48e8-8df4-2de32a84f65d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:01.432740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 166/323] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:27:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:01.438103 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfe942-86dd-48e8-8df4-2de32a84f65d None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:01.492450 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 158/324] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:27:01.515849 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2654b1-8b6a-4a49-b93d-c4f4ce965118 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:01.539735 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:01.543555 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfe942-86dd-48e8-8df4-2de32a84f65d None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:01.577632 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec9874-621e-4a2d-99f0-6508cc0a87e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:01.578502 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec9874-621e-4a2d-99f0-6508cc0a87e6 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:01.578789 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec9874-621e-4a2d-99f0-6508cc0a87e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:01.579067 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec9874-621e-4a2d-99f0-6508cc0a87e6 None admin] PATH_INFO: `/v3/users/fc95004e6fe54619ab7cd139088cc591` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:01.580260 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec9874-621e-4a2d-99f0-6508cc0a87e6 None admin] RBAC: Authorizing `identity:get_user(user_id=fc95004e6fe54619ab7cd139088cc591)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:01.581732 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec9874-621e-4a2d-99f0-6508cc0a87e6 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:01.583363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 167/325] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:27:01 2022] GET /identity/v3/users/fc95004e6fe54619ab7cd139088cc591 => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:01.584522 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf3086-7489-4989-8d7e-8740ac18edf5 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:01.584833 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf3086-7489-4989-8d7e-8740ac18edf5 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:01.585120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf3086-7489-4989-8d7e-8740ac18edf5 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:01.586353 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 168/326] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:01.589193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf3086-7489-4989-8d7e-8740ac18edf5 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:01.614957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2654b1-8b6a-4a49-b93d-c4f4ce965118 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:01.649946 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8efbf-afc7-4b97-be68-3673b8bab4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:01.650734 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8efbf-afc7-4b97-be68-3673b8bab4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:01.651021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8efbf-afc7-4b97-be68-3673b8bab4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:01.651304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8efbf-afc7-4b97-be68-3673b8bab4f1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:01.661805 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8efbf-afc7-4b97-be68-3673b8bab4f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:01.670527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-b6e8efbf-afc7-4b97-be68-3673b8bab4f1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:27:01.670527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:01.670527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:01.670527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:01.670527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:01.670527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:01.670527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:01.670527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:01.670527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:01.670527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:01.670527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:01.670527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:01.670527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:01.670527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:01.670527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:01.670527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:01.671459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:01.671459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:01.671459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:01.671459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:01.671459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:01.671459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:01.671459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:01.671459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:01.671459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:01.671459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:01.671459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:01.671459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:01.671459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:01.671459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:01.671459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:01.671459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:01.672250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:01.672250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:01.672250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:01.672250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:01.672250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:01.672250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:01.672250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:01.672250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:01.672250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:01.672250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:01.672250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:27:01.672250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:01.672250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 159/327] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:27:01.672962 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708765f-e6a7-4c6b-9c84-6acc3beb3be7 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:01.673423 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708765f-e6a7-4c6b-9c84-6acc3beb3be7 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:01.673423 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708765f-e6a7-4c6b-9c84-6acc3beb3be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:01.678688 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:01.691544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-9708765f-e6a7-4c6b-9c84-6acc3beb3be7 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:01.692756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf3086-7489-4989-8d7e-8740ac18edf5 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:01.692870 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-9708765f-e6a7-4c6b-9c84-6acc3beb3be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:01.726300 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c183e-f1f0-4d04-9201-ec82c84e6f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:01.726948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c183e-f1f0-4d04-9201-ec82c84e6f7e None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:01.727223 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c183e-f1f0-4d04-9201-ec82c84e6f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:01.727503 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c183e-f1f0-4d04-9201-ec82c84e6f7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:01.729005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c183e-f1f0-4d04-9201-ec82c84e6f7e None admin] RBAC: Adding query filter params (role.id=1305747c745d49dc9a9718b7aa8aa4a9, user.id=fc95004e6fe54619ab7cd139088cc591, scope.system=all) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:01.729322 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c183e-f1f0-4d04-9201-ec82c84e6f7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:01.730742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c183e-f1f0-4d04-9201-ec82c84e6f7e None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:01.741892 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 169/328] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Jun 23 22:27:01 2022] GET /identity/v3/role_assignments?user.id=fc95004e6fe54619ab7cd139088cc591&scope.system=all&role.id=1305747c745d49dc9a9718b7aa8aa4a9 => generated 510 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:27:01.745538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c183e-f1f0-4d04-9201-ec82c84e6f7e None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:01.769408 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 160/329] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:01.788072 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:01.792928 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708765f-e6a7-4c6b-9c84-6acc3beb3be7 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:01.873454 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c183e-f1f0-4d04-9201-ec82c84e6f7e None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:01.909181 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708765f-e6a7-4c6b-9c84-6acc3beb3be7 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:01.920708 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee00cf-6e74-497a-9fd0-c10954a779cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:01.921394 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee00cf-6e74-497a-9fd0-c10954a779cd None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:01.921687 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee00cf-6e74-497a-9fd0-c10954a779cd None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:01.921975 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee00cf-6e74-497a-9fd0-c10954a779cd None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:01.922740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee00cf-6e74-497a-9fd0-c10954a779cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:01.923047 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee00cf-6e74-497a-9fd0-c10954a779cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:01.924426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee00cf-6e74-497a-9fd0-c10954a779cd None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:01.934352 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 170/330] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:27:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:01.939768 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee00cf-6e74-497a-9fd0-c10954a779cd None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:01.946122 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:01.951332 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67731551-0698-440a-80f1-d0d3ee93762a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:01.952134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67731551-0698-440a-80f1-d0d3ee93762a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:01.952410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67731551-0698-440a-80f1-d0d3ee93762a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:01.958474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67731551-0698-440a-80f1-d0d3ee93762a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:01.958474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67731551-0698-440a-80f1-d0d3ee93762a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:01.958474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67731551-0698-440a-80f1-d0d3ee93762a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:01.958474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 161/331] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:27:01.962885 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67731551-0698-440a-80f1-d0d3ee93762a None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:02.028443 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:02.040977 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee00cf-6e74-497a-9fd0-c10954a779cd None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:02.062903 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67731551-0698-440a-80f1-d0d3ee93762a None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:02.074487 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db325f6-2c44-472f-8c45-b20d1e63e813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:02.075360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db325f6-2c44-472f-8c45-b20d1e63e813 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:02.075649 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db325f6-2c44-472f-8c45-b20d1e63e813 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:02.075930 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db325f6-2c44-472f-8c45-b20d1e63e813 None admin] PATH_INFO: `/v3/users/faf96ff3ffe54a2aa420bb5dba3cc2b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:02.077279 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db325f6-2c44-472f-8c45-b20d1e63e813 None admin] RBAC: Authorizing `identity:get_user(user_id=faf96ff3ffe54a2aa420bb5dba3cc2b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:02.078716 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db325f6-2c44-472f-8c45-b20d1e63e813 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:02.080490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 171/332] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:27:01 2022] GET /identity/v3/users/faf96ff3ffe54a2aa420bb5dba3cc2b4 => generated 293 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:27:02.086023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db325f6-2c44-472f-8c45-b20d1e63e813 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:02.096492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecdb16-fe7e-4b39-91f9-5155bb282228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:02.097189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecdb16-fe7e-4b39-91f9-5155bb282228 None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:02.097473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecdb16-fe7e-4b39-91f9-5155bb282228 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:02.097764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecdb16-fe7e-4b39-91f9-5155bb282228 None admin] PATH_INFO: `/v3/groups` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:02.098452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ecdb16-fe7e-4b39-91f9-5155bb282228 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:02.099863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ecdb16-fe7e-4b39-91f9-5155bb282228 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:02.167572 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 162/333] 10.222.0.33 () {60 vars in 1134 bytes} [Thu Jun 23 22:27:01 2022] POST /identity/v3/groups => generated 226 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:27:02.184878 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db325f6-2c44-472f-8c45-b20d1e63e813 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:02.220875 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8392d-98f9-4853-bc45-17d5a126814b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:02.221771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8392d-98f9-4853-bc45-17d5a126814b None admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:02.222058 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8392d-98f9-4853-bc45-17d5a126814b None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:02.222340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8392d-98f9-4853-bc45-17d5a126814b None admin] PATH_INFO: `/v3/system/users/faf96ff3ffe54a2aa420bb5dba3cc2b4/roles/917b4049bffd4db282dfe80ac37538be` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:02.223959 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8392d-98f9-4853-bc45-17d5a126814b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=faf96ff3ffe54a2aa420bb5dba3cc2b4, role_id=917b4049bffd4db282dfe80ac37538be)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:02.225427 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8392d-98f9-4853-bc45-17d5a126814b None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:02.283331 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 172/334] 10.222.0.33 () {58 vars in 1334 bytes} [Thu Jun 23 22:27:02 2022] PUT /identity/v3/system/users/faf96ff3ffe54a2aa420bb5dba3cc2b4/roles/917b4049bffd4db282dfe80ac37538be => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:27:04.785576 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ac823-9535-4e3a-9ee1-fbaaa5e30958 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:04.786549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ac823-9535-4e3a-9ee1-fbaaa5e30958 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:04.787068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ac823-9535-4e3a-9ee1-fbaaa5e30958 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:04.788956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 163/335] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:04.795301 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5432a-e54b-4437-b437-15250b94d3ef None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:04.795738 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5432a-e54b-4437-b437-15250b94d3ef None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:04.795951 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5432a-e54b-4437-b437-15250b94d3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:04.820734 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-fba5432a-e54b-4437-b437-15250b94d3ef None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:04.822003 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-fba5432a-e54b-4437-b437-15250b94d3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:04.842217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:04.893007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2542a1c-c010-498b-82c6-0c3f776c3264 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:04.893663 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2542a1c-c010-498b-82c6-0c3f776c3264 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:04.894147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2542a1c-c010-498b-82c6-0c3f776c3264 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:04.895787 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 164/336] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:04.906022 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 173/337] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:04.908773 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805adde3-9ea3-48ab-ac2a-da5dc125fcc8 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:04.909293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805adde3-9ea3-48ab-ac2a-da5dc125fcc8 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:04.909792 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805adde3-9ea3-48ab-ac2a-da5dc125fcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:04.942491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-805adde3-9ea3-48ab-ac2a-da5dc125fcc8 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:04.943096 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66003e3c-8eef-48e7-94fe-706089624db2 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:04.943555 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66003e3c-8eef-48e7-94fe-706089624db2 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:04.943980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66003e3c-8eef-48e7-94fe-706089624db2 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:04.944297 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-805adde3-9ea3-48ab-ac2a-da5dc125fcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:04.945946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 174/338] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:04.952274 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c90ff-72fe-4641-933a-42e0215adbe1 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:04.952814 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c90ff-72fe-4641-933a-42e0215adbe1 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:04.953305 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c90ff-72fe-4641-933a-42e0215adbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:04.981643 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-397c90ff-72fe-4641-933a-42e0215adbe1 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:04.985836 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-397c90ff-72fe-4641-933a-42e0215adbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:05.057673 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:05.072478 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 165/339] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:05.073566 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805adde3-9ea3-48ab-ac2a-da5dc125fcc8 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:05.123797 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 175/340] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:05.140261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c90ff-72fe-4641-933a-42e0215adbe1 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:05.207850 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805adde3-9ea3-48ab-ac2a-da5dc125fcc8 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:05.264942 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:05.267116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b2dd2-69a4-4194-9230-7d9c5b72d826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:05.267971 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b2dd2-69a4-4194-9230-7d9c5b72d826 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:05.268225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b2dd2-69a4-4194-9230-7d9c5b72d826 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:05.268499 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b2dd2-69a4-4194-9230-7d9c5b72d826 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:05.293875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b2dd2-69a4-4194-9230-7d9c5b72d826 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:05.302052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-682b2dd2-69a4-4194-9230-7d9c5b72d826 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:27:05.302052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:05.302052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:05.302052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:05.302052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:05.302052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:05.302052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:05.302052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:05.302052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:05.302052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:05.302052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:05.302052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:05.302052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:05.302052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:05.302052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:05.302052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:05.303031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:05.303031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:05.303031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:05.303031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:05.303031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:05.303031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:05.303031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:05.303031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:05.303031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:05.303031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:05.303031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:05.303031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:05.303031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:05.303031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:05.303031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:05.303031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:05.303823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:05.303823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:05.303823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:05.303823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:05.303823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:05.303823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:05.303823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:05.303823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:05.303823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:05.303823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:05.303823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:27:05.303823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:05.303823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 166/341] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:27:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:05.304498 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c90ff-72fe-4641-933a-42e0215adbe1 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:05.304570 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b2dd2-69a4-4194-9230-7d9c5b72d826 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:05.377290 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49419f4d-bd38-42dc-9043-a5e41412cef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:05.378811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49419f4d-bd38-42dc-9043-a5e41412cef6 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:05.379309 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49419f4d-bd38-42dc-9043-a5e41412cef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:05.379820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49419f4d-bd38-42dc-9043-a5e41412cef6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:05.410194 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:05.416456 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49419f4d-bd38-42dc-9043-a5e41412cef6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:05.428720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-49419f4d-bd38-42dc-9043-a5e41412cef6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:27:05.428720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:05.428720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:05.428720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:05.428720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:05.428720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:05.428720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:05.428720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:05.428720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:05.428720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:05.428720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:05.428720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:05.428720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:05.428720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:05.428720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:05.428720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:05.432704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:05.432704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:05.432704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:05.432704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:05.432704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:05.432704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:05.432704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:05.432704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:05.432704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:05.432704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:05.432704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:05.432704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:05.432704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:05.432704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:05.432704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:05.432704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:05.433986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:05.433986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:05.433986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:05.433986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:05.433986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:05.433986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:05.433986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:05.433986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:05.433986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:05.433986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:05.433986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:27:05.433986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:05.433986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 176/342] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:27:05.433986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49419f4d-bd38-42dc-9043-a5e41412cef6 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:05.467923 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b2dd2-69a4-4194-9230-7d9c5b72d826 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:05.507465 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc59d14-2e80-4eaf-9d36-62ef56bbbdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:05.508349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc59d14-2e80-4eaf-9d36-62ef56bbbdf7 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:05.508868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc59d14-2e80-4eaf-9d36-62ef56bbbdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:05.509148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc59d14-2e80-4eaf-9d36-62ef56bbbdf7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:05.522217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc59d14-2e80-4eaf-9d36-62ef56bbbdf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:05.530868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-8dc59d14-2e80-4eaf-9d36-62ef56bbbdf7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:27:05.530868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:05.530868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:05.530868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:05.530868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:05.530868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:05.530868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:05.530868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:05.530868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:05.530868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:05.530868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:05.530868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:05.530868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:05.530868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:05.530868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:05.530868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:05.531882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:05.531882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:05.531882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:05.531882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:05.531882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:05.531882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:05.531882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:05.531882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:05.531882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:05.531882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:05.531882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:05.531882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:05.531882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:05.531882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:05.531882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:05.531882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:05.532740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:05.532740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:05.532740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:05.532740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:05.532740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:05.532740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:05.532740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:05.532740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:05.532740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:05.532740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:05.532740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:27:05.532740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:05.532740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 167/343] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:05.534199 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc59d14-2e80-4eaf-9d36-62ef56bbbdf7 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:05.577171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49419f4d-bd38-42dc-9043-a5e41412cef6 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:05.621277 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18736b9f-a2a7-4303-9466-3a10655f0292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:05.621969 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18736b9f-a2a7-4303-9466-3a10655f0292 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:05.622256 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18736b9f-a2a7-4303-9466-3a10655f0292 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:05.622541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18736b9f-a2a7-4303-9466-3a10655f0292 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:05.623305 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18736b9f-a2a7-4303-9466-3a10655f0292 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:05.623721 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18736b9f-a2a7-4303-9466-3a10655f0292 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:05.625251 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18736b9f-a2a7-4303-9466-3a10655f0292 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:05.639926 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 177/344] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:05.641362 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2eda3-514a-4133-bda5-f40a2de43607 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:05.641655 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2eda3-514a-4133-bda5-f40a2de43607 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:05.641928 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2eda3-514a-4133-bda5-f40a2de43607 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:05.643168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 178/345] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:05.643845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc59d14-2e80-4eaf-9d36-62ef56bbbdf7 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:05.643956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2eda3-514a-4133-bda5-f40a2de43607 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:05.655536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:05.656733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:05.690439 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cc974-073a-42aa-a260-4d9e73cfe8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:05.691128 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cc974-073a-42aa-a260-4d9e73cfe8ad None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:05.691415 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cc974-073a-42aa-a260-4d9e73cfe8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:05.691696 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cc974-073a-42aa-a260-4d9e73cfe8ad None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:05.692454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cc974-073a-42aa-a260-4d9e73cfe8ad None admin] RBAC: Adding query filter params (name=member) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:05.692845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cc974-073a-42aa-a260-4d9e73cfe8ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:05.694244 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cc974-073a-42aa-a260-4d9e73cfe8ad None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:05.704366 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 168/346] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:27:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:05.705299 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cc974-073a-42aa-a260-4d9e73cfe8ad None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:05.807236 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2eda3-514a-4133-bda5-f40a2de43607 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:05.831639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cc974-073a-42aa-a260-4d9e73cfe8ad None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:05.853487 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafcae9-bde7-4c3f-accf-448e21f74acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:05.854167 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafcae9-bde7-4c3f-accf-448e21f74acc None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:05.854452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafcae9-bde7-4c3f-accf-448e21f74acc None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:05.854726 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafcae9-bde7-4c3f-accf-448e21f74acc None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:05.855478 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafcae9-bde7-4c3f-accf-448e21f74acc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:05.855794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafcae9-bde7-4c3f-accf-448e21f74acc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:05.857297 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafcae9-bde7-4c3f-accf-448e21f74acc None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:05.867600 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 179/347] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:27:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:05.869700 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271871b-fb9e-4ffc-b77a-41128fd7bd7b None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:05.870053 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271871b-fb9e-4ffc-b77a-41128fd7bd7b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:05.870348 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271871b-fb9e-4ffc-b77a-41128fd7bd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:05.871677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3b68b-b548-4693-a9f7-695cf1608b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:05.872499 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3b68b-b548-4693-a9f7-695cf1608b08 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:05.873095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3b68b-b548-4693-a9f7-695cf1608b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:05.873550 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3b68b-b548-4693-a9f7-695cf1608b08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:05.890162 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-7271871b-fb9e-4ffc-b77a-41128fd7bd7b None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:05.891439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-7271871b-fb9e-4ffc-b77a-41128fd7bd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:05.937339 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:05.969921 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3b68b-b548-4693-a9f7-695cf1608b08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:05.970667 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:05.973473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3b68b-b548-4693-a9f7-695cf1608b08 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:05.980781 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 180/348] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:05.983234 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271871b-fb9e-4ffc-b77a-41128fd7bd7b None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:06.097549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271871b-fb9e-4ffc-b77a-41128fd7bd7b None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:06.121587 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 169/349] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:06.124091 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e92330-acdf-4c6b-b8df-b19e6d278845 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:06.124657 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e92330-acdf-4c6b-b8df-b19e6d278845 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:06.125337 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e92330-acdf-4c6b-b8df-b19e6d278845 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:06.127492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 170/350] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:06.129039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e92330-acdf-4c6b-b8df-b19e6d278845 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:06.143129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5005641-b430-4a58-8a6e-a7c57fcaf31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:06.143733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5005641-b430-4a58-8a6e-a7c57fcaf31c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:06.144024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5005641-b430-4a58-8a6e-a7c57fcaf31c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:06.144308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5005641-b430-4a58-8a6e-a7c57fcaf31c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:06.222146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:06.230045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:06.234804 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5005641-b430-4a58-8a6e-a7c57fcaf31c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:06.236308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5005641-b430-4a58-8a6e-a7c57fcaf31c None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:06.277626 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e92330-acdf-4c6b-b8df-b19e6d278845 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:06.321762 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d222a-b06d-46ab-8226-5ad657e6c09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:06.322667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d222a-b06d-46ab-8226-5ad657e6c09d None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:06.322957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d222a-b06d-46ab-8226-5ad657e6c09d None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:06.323234 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d222a-b06d-46ab-8226-5ad657e6c09d None admin] PATH_INFO: `/v3/users/faf96ff3ffe54a2aa420bb5dba3cc2b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:06.324976 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d222a-b06d-46ab-8226-5ad657e6c09d None admin] RBAC: Authorizing `identity:get_user(user_id=faf96ff3ffe54a2aa420bb5dba3cc2b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:06.326705 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d222a-b06d-46ab-8226-5ad657e6c09d None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:06.328676 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 171/351] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:27:06 2022] GET /identity/v3/users/faf96ff3ffe54a2aa420bb5dba3cc2b4 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:06.329645 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d222a-b06d-46ab-8226-5ad657e6c09d None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:06.380055 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 181/352] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:06.381257 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5005641-b430-4a58-8a6e-a7c57fcaf31c None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:06.464689 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d222a-b06d-46ab-8226-5ad657e6c09d None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:06.509621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca00241-99bd-43ca-a491-c7fa80b59f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:06.510315 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:06.511705 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca00241-99bd-43ca-a491-c7fa80b59f53 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:06.512204 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca00241-99bd-43ca-a491-c7fa80b59f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:06.512996 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca00241-99bd-43ca-a491-c7fa80b59f53 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:06.518603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5005641-b430-4a58-8a6e-a7c57fcaf31c None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:06.523788 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:06.529655 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca00241-99bd-43ca-a491-c7fa80b59f53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:06.537815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-9ca00241-99bd-43ca-a491-c7fa80b59f53 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:27:06.537815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:06.537815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:06.537815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:06.537815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:06.537815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:06.537815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:06.537815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:06.537815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:06.537815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:06.537815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:06.537815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:06.537815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:06.537815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:06.537815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:06.537815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:06.538724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:06.538724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:06.538724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:06.538724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:06.538724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:06.538724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:06.538724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:06.538724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:06.538724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:06.538724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:06.538724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:06.538724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:06.538724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:06.538724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:06.538724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:06.538724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:06.539510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:06.539510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:06.539510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:06.539510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:06.539510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:06.539510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:06.539510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:06.539510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:06.539510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:06.539510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:06.539510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:27:06.539510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:06.539510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 172/353] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:06.542713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49ccf3-bb2e-4445-b52d-daef4a4355e0 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:06.543007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49ccf3-bb2e-4445-b52d-daef4a4355e0 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:06.543550 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49ccf3-bb2e-4445-b52d-daef4a4355e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:06.561319 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-ca49ccf3-bb2e-4445-b52d-daef4a4355e0 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:06.562225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ca49ccf3-bb2e-4445-b52d-daef4a4355e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:06.566409 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a781675-abba-4040-84ce-a3169ddba733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:06.567075 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a781675-abba-4040-84ce-a3169ddba733 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:06.567364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a781675-abba-4040-84ce-a3169ddba733 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:06.567645 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a781675-abba-4040-84ce-a3169ddba733 None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:06.568928 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a781675-abba-4040-84ce-a3169ddba733 None admin] RBAC: Authorizing `identity:get_project(project_id=24e47f2dd0c74c218f4fd2059483f785)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:06.570373 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a781675-abba-4040-84ce-a3169ddba733 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:06.576661 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 182/354] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:27:06 2022] GET /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:06.577527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a781675-abba-4040-84ce-a3169ddba733 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:06.645401 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 173/355] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:06.646250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49ccf3-bb2e-4445-b52d-daef4a4355e0 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:06.700838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a781675-abba-4040-84ce-a3169ddba733 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:06.744689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b768a-bc18-4468-bd3c-8ab24e1bb557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:06.745910 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b768a-bc18-4468-bd3c-8ab24e1bb557 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:06.746727 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b768a-bc18-4468-bd3c-8ab24e1bb557 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:06.747016 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b768a-bc18-4468-bd3c-8ab24e1bb557 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:06.747871 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b768a-bc18-4468-bd3c-8ab24e1bb557 None admin] RBAC: Adding query filter params (role.id=917b4049bffd4db282dfe80ac37538be, user.id=faf96ff3ffe54a2aa420bb5dba3cc2b4, scope.system=all) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:06.748190 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b768a-bc18-4468-bd3c-8ab24e1bb557 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:06.749744 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b768a-bc18-4468-bd3c-8ab24e1bb557 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:06.764597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49ccf3-bb2e-4445-b52d-daef4a4355e0 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:06.766164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 183/356] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Jun 23 22:27:06 2022] GET /identity/v3/role_assignments?user.id=faf96ff3ffe54a2aa420bb5dba3cc2b4&scope.system=all&role.id=917b4049bffd4db282dfe80ac37538be => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:06.766486 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fca06-b20a-4622-9ea0-fd901d8b9030 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:06.766779 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fca06-b20a-4622-9ea0-fd901d8b9030 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:06.768750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fca06-b20a-4622-9ea0-fd901d8b9030 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:06.768750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 184/357] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:06.769748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fca06-b20a-4622-9ea0-fd901d8b9030 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:06.801642 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:06.804295 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:06.811386 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b32d58-82e1-486e-bfab-4c2aa46c57b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:06.812288 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b32d58-82e1-486e-bfab-4c2aa46c57b4 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:06.812664 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b32d58-82e1-486e-bfab-4c2aa46c57b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:06.812883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b32d58-82e1-486e-bfab-4c2aa46c57b4 None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:06.814730 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b32d58-82e1-486e-bfab-4c2aa46c57b4 None admin] RBAC: Authorizing `identity:get_project(project_id=24e47f2dd0c74c218f4fd2059483f785)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:06.816231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b32d58-82e1-486e-bfab-4c2aa46c57b4 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:06.818064 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 174/358] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:27:06 2022] GET /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:06.818856 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b32d58-82e1-486e-bfab-4c2aa46c57b4 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:06.922717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fca06-b20a-4622-9ea0-fd901d8b9030 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:06.945195 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b32d58-82e1-486e-bfab-4c2aa46c57b4 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:06.967014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c6627-b0eb-4f7b-bd95-b207525d7b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:06.967708 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c6627-b0eb-4f7b-bd95-b207525d7b13 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:06.967995 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c6627-b0eb-4f7b-bd95-b207525d7b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:06.968274 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c6627-b0eb-4f7b-bd95-b207525d7b13 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:06.969527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c6627-b0eb-4f7b-bd95-b207525d7b13 None admin] RBAC: Adding query filter params (name=member) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:06.969842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c6627-b0eb-4f7b-bd95-b207525d7b13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:06.971668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c6627-b0eb-4f7b-bd95-b207525d7b13 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:06.984289 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 185/359] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:27:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:06.985027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c6627-b0eb-4f7b-bd95-b207525d7b13 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:06.990927 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7d3be-e976-4613-9b4f-903da6e00193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:06.991805 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7d3be-e976-4613-9b4f-903da6e00193 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:06.992120 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7d3be-e976-4613-9b4f-903da6e00193 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:06.992374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7d3be-e976-4613-9b4f-903da6e00193 None admin] PATH_INFO: `/v3/groups/fe95c2a6f34746a8b469bf15af149d71` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:06.999967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e7d3be-e976-4613-9b4f-903da6e00193 None admin] RBAC: Authorizing `identity:get_group(group_id=fe95c2a6f34746a8b469bf15af149d71)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:07.001414 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e7d3be-e976-4613-9b4f-903da6e00193 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:07.003123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 175/360] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 22:27:06 2022] GET /identity/v3/groups/fe95c2a6f34746a8b469bf15af149d71 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:07.004315 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c0884-c50e-4fcb-b20d-b69fc12679f8 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:07.006380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c0884-c50e-4fcb-b20d-b69fc12679f8 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:07.006380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c0884-c50e-4fcb-b20d-b69fc12679f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:07.025278 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-360c0884-c50e-4fcb-b20d-b69fc12679f8 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:07.026491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-360c0884-c50e-4fcb-b20d-b69fc12679f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:07.082626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:07.093634 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:07.105861 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 176/361] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:07.106385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c6627-b0eb-4f7b-bd95-b207525d7b13 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:07.106674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c0884-c50e-4fcb-b20d-b69fc12679f8 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:07.147089 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae4ee4-0263-42c6-ab8b-fd0ed7fad60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:07.147910 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae4ee4-0263-42c6-ab8b-fd0ed7fad60f None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:07.148193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae4ee4-0263-42c6-ab8b-fd0ed7fad60f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:07.148475 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae4ee4-0263-42c6-ab8b-fd0ed7fad60f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:07.163790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae4ee4-0263-42c6-ab8b-fd0ed7fad60f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:07.173042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-e5ae4ee4-0263-42c6-ab8b-fd0ed7fad60f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:27:07.173042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:07.173042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:07.173042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:07.173042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:07.173042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:07.173042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:07.173042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:07.173042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:07.173042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:07.173042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:07.173042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:07.173042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:07.173042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:07.173042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:07.173042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:07.173948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:07.173948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:07.173948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:07.173948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:07.173948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:07.173948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:07.173948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:07.173948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:07.173948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:07.173948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:07.173948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:07.173948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:07.173948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:07.173948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:07.173948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:07.173948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:07.174740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:07.174740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:07.174740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:07.174740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:07.174740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:07.174740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:07.174740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:07.174740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:07.174740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:07.174740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:07.174740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:27:07.174740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:07.174740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 186/362] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:27:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:07.175417 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae4ee4-0263-42c6-ab8b-fd0ed7fad60f None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:07.232242 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c0884-c50e-4fcb-b20d-b69fc12679f8 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:07.275310 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22532cfe-3141-4d1c-8f47-4a44b8bcd098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:07.276161 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22532cfe-3141-4d1c-8f47-4a44b8bcd098 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:07.277351 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22532cfe-3141-4d1c-8f47-4a44b8bcd098 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:07.277351 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22532cfe-3141-4d1c-8f47-4a44b8bcd098 None admin] PATH_INFO: `/v3/groups/fe95c2a6f34746a8b469bf15af149d71` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:07.278823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22532cfe-3141-4d1c-8f47-4a44b8bcd098 None admin] RBAC: Authorizing `identity:get_group(group_id=fe95c2a6f34746a8b469bf15af149d71)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:07.280290 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22532cfe-3141-4d1c-8f47-4a44b8bcd098 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:07.288485 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 177/363] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 22:27:07 2022] GET /identity/v3/groups/fe95c2a6f34746a8b469bf15af149d71 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:07.288485 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22532cfe-3141-4d1c-8f47-4a44b8bcd098 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:07.301326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae4ee4-0263-42c6-ab8b-fd0ed7fad60f None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:07.341863 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815afa6-c968-4c8f-bb49-d630d2235b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:07.342524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815afa6-c968-4c8f-bb49-d630d2235b12 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:07.342808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815afa6-c968-4c8f-bb49-d630d2235b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:07.343083 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815afa6-c968-4c8f-bb49-d630d2235b12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:07.358694 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:07.361103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:07.407441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22532cfe-3141-4d1c-8f47-4a44b8bcd098 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:07.425822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5815afa6-c968-4c8f-bb49-d630d2235b12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:07.427313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5815afa6-c968-4c8f-bb49-d630d2235b12 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:07.446870 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efbc224-52c3-4457-87a2-1f4568cb9603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:07.448287 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbc224-52c3-4457-87a2-1f4568cb9603 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:07.448287 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbc224-52c3-4457-87a2-1f4568cb9603 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:07.448486 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbc224-52c3-4457-87a2-1f4568cb9603 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:07.449369 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efbc224-52c3-4457-87a2-1f4568cb9603 None admin] RBAC: Adding query filter params (group.id=fe95c2a6f34746a8b469bf15af149d71, role.id=c3ad3cf6be504e35aaec10b7b9f3f34d, scope.project.id=24e47f2dd0c74c218f4fd2059483f785) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:07.449687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efbc224-52c3-4457-87a2-1f4568cb9603 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:07.451141 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efbc224-52c3-4457-87a2-1f4568cb9603 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:07.457167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 178/364] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 23 22:27:07 2022] GET /identity/v3/role_assignments?group.id=fe95c2a6f34746a8b469bf15af149d71&scope.project.id=24e47f2dd0c74c218f4fd2059483f785&role.id=c3ad3cf6be504e35aaec10b7b9f3f34d => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:07.457982 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efbc224-52c3-4457-87a2-1f4568cb9603 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:07.569484 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 187/365] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:07.570254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815afa6-c968-4c8f-bb49-d630d2235b12 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:07.609805 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efbc224-52c3-4457-87a2-1f4568cb9603 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:07.629507 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:07.638272 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:07.655162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684b8c4-c565-480f-abaa-d1a737f70d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:07.656250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684b8c4-c565-480f-abaa-d1a737f70d0d None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:07.656726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684b8c4-c565-480f-abaa-d1a737f70d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:07.657157 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684b8c4-c565-480f-abaa-d1a737f70d0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:07.670731 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684b8c4-c565-480f-abaa-d1a737f70d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:07.681296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-9684b8c4-c565-480f-abaa-d1a737f70d0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:27:07.681296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:07.681296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:07.681296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:07.681296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:07.681296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:07.681296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:07.681296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:07.681296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:07.681296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:07.681296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:07.681296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:07.681296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:07.681296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:07.681296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:07.681296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:07.682409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:07.682409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:07.682409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:07.682409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:07.682409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:07.682409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:07.682409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:07.682409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:07.682409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:07.682409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:07.682409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:07.682409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:07.682409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:07.682409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:07.682409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:07.682409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:07.683406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:07.683406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:07.683406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:07.683406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:07.683406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:07.683406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:07.683406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:07.683406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:07.683406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:07.683406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:07.683406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:27:07.683406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:07.685545 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 179/366] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:27:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:07.686426 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684b8c4-c565-480f-abaa-d1a737f70d0d None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:07.702215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815afa6-c968-4c8f-bb49-d630d2235b12 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:07.746006 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c665eab-0fb3-4f98-a931-5c261cd18366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:07.747140 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c665eab-0fb3-4f98-a931-5c261cd18366 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:07.747428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c665eab-0fb3-4f98-a931-5c261cd18366 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:07.747703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c665eab-0fb3-4f98-a931-5c261cd18366 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:07.748835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c665eab-0fb3-4f98-a931-5c261cd18366 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:07.749147 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c665eab-0fb3-4f98-a931-5c261cd18366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:07.750520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c665eab-0fb3-4f98-a931-5c261cd18366 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:07.761247 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 188/367] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:07.762097 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c665eab-0fb3-4f98-a931-5c261cd18366 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:07.819432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684b8c4-c565-480f-abaa-d1a737f70d0d None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:07.858842 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a5872-a849-42ab-9ff9-801cf786c43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:07.859434 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a5872-a849-42ab-9ff9-801cf786c43b None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:07.859724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a5872-a849-42ab-9ff9-801cf786c43b None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:07.860008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a5872-a849-42ab-9ff9-801cf786c43b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:07.860893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a5872-a849-42ab-9ff9-801cf786c43b None admin] RBAC: Adding query filter params (group.id=fe95c2a6f34746a8b469bf15af149d71, role.id=1305747c745d49dc9a9718b7aa8aa4a9, scope.project.id=24e47f2dd0c74c218f4fd2059483f785) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:07.861216 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a5872-a849-42ab-9ff9-801cf786c43b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:07.862670 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a5872-a849-42ab-9ff9-801cf786c43b None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:07.871578 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 180/368] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 23 22:27:07 2022] GET /identity/v3/role_assignments?group.id=fe95c2a6f34746a8b469bf15af149d71&scope.project.id=24e47f2dd0c74c218f4fd2059483f785&role.id=1305747c745d49dc9a9718b7aa8aa4a9 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:07.872150 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a5872-a849-42ab-9ff9-801cf786c43b None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:07.895156 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c665eab-0fb3-4f98-a931-5c261cd18366 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:07.900509 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:07.937143 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:07.945138 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfcbf43-dbe8-4926-b841-dea67d16918f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:07.947216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcbf43-dbe8-4926-b841-dea67d16918f None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:07.947865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcbf43-dbe8-4926-b841-dea67d16918f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:07.948317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcbf43-dbe8-4926-b841-dea67d16918f None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:07.950459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfcbf43-dbe8-4926-b841-dea67d16918f None admin] RBAC: Authorizing `identity:get_project(project_id=debcfc4c17a34d8a80f13016124abfaf)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:07.952632 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfcbf43-dbe8-4926-b841-dea67d16918f None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:07.955323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 189/369] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:27:07 2022] GET /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:07.956989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfcbf43-dbe8-4926-b841-dea67d16918f None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:07.997715 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a5872-a849-42ab-9ff9-801cf786c43b None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:08.040668 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731244e-bb20-481e-87fd-6a4400473616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:08.041371 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731244e-bb20-481e-87fd-6a4400473616 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:08.041679 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731244e-bb20-481e-87fd-6a4400473616 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:08.042308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731244e-bb20-481e-87fd-6a4400473616 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:08.044006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731244e-bb20-481e-87fd-6a4400473616 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:08.044482 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731244e-bb20-481e-87fd-6a4400473616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:08.046059 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731244e-bb20-481e-87fd-6a4400473616 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:08.057940 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 181/370] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:08.058844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731244e-bb20-481e-87fd-6a4400473616 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:08.099359 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfcbf43-dbe8-4926-b841-dea67d16918f None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:08.137099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e886a-5e51-418e-b9c2-7a59cf2ac298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:08.137678 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e886a-5e51-418e-b9c2-7a59cf2ac298 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:08.138070 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e886a-5e51-418e-b9c2-7a59cf2ac298 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:08.138355 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e886a-5e51-418e-b9c2-7a59cf2ac298 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:08.186252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:08.194055 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731244e-bb20-481e-87fd-6a4400473616 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:08.208705 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:08.217719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e886a-5e51-418e-b9c2-7a59cf2ac298 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:08.219263 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e886a-5e51-418e-b9c2-7a59cf2ac298 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:08.238400 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e28878-c553-4b28-85e2-af1be2d5474f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:08.239563 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e28878-c553-4b28-85e2-af1be2d5474f None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:08.240110 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e28878-c553-4b28-85e2-af1be2d5474f None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:08.240601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e28878-c553-4b28-85e2-af1be2d5474f None admin] PATH_INFO: `/v3/groups/fe95c2a6f34746a8b469bf15af149d71` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:08.242262 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e28878-c553-4b28-85e2-af1be2d5474f None admin] RBAC: Authorizing `identity:get_group(group_id=fe95c2a6f34746a8b469bf15af149d71)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:08.244225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e28878-c553-4b28-85e2-af1be2d5474f None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:08.246514 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 182/371] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 22:27:08 2022] GET /identity/v3/groups/fe95c2a6f34746a8b469bf15af149d71 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:08.247641 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e28878-c553-4b28-85e2-af1be2d5474f None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:08.338790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 190/372] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:08.339450 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e886a-5e51-418e-b9c2-7a59cf2ac298 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:08.357875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e28878-c553-4b28-85e2-af1be2d5474f None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:08.393594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb8e65-98ab-40a9-adf2-fc1a9fa72062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:08.394600 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb8e65-98ab-40a9-adf2-fc1a9fa72062 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:08.395095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb8e65-98ab-40a9-adf2-fc1a9fa72062 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:08.395573 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb8e65-98ab-40a9-adf2-fc1a9fa72062 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:08.415188 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:08.444755 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:08.450275 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e886a-5e51-418e-b9c2-7a59cf2ac298 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:08.467784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb8e65-98ab-40a9-adf2-fc1a9fa72062 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:08.469406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb8e65-98ab-40a9-adf2-fc1a9fa72062 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:08.496416 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9387f2a-9b01-442e-b250-d8128c608437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:08.497397 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9387f2a-9b01-442e-b250-d8128c608437 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:08.497688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9387f2a-9b01-442e-b250-d8128c608437 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:08.497969 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9387f2a-9b01-442e-b250-d8128c608437 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:08.499062 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9387f2a-9b01-442e-b250-d8128c608437 None admin] RBAC: Adding query filter params (group.id=fe95c2a6f34746a8b469bf15af149d71, role.id=1305747c745d49dc9a9718b7aa8aa4a9, scope.project.id=debcfc4c17a34d8a80f13016124abfaf) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:08.499407 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9387f2a-9b01-442e-b250-d8128c608437 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:08.501570 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9387f2a-9b01-442e-b250-d8128c608437 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:08.510010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 191/373] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 23 22:27:08 2022] GET /identity/v3/role_assignments?group.id=fe95c2a6f34746a8b469bf15af149d71&scope.project.id=debcfc4c17a34d8a80f13016124abfaf&role.id=1305747c745d49dc9a9718b7aa8aa4a9 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:08.510923 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9387f2a-9b01-442e-b250-d8128c608437 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:08.618834 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 183/374] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:08.625711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb8e65-98ab-40a9-adf2-fc1a9fa72062 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:08.723005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9387f2a-9b01-442e-b250-d8128c608437 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:08.760943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:08.762868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d15fc-b54a-47a4-98af-ed05f10d9e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:08.763812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d15fc-b54a-47a4-98af-ed05f10d9e30 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:08.764165 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d15fc-b54a-47a4-98af-ed05f10d9e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:08.764509 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d15fc-b54a-47a4-98af-ed05f10d9e30 None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:08.766093 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d15fc-b54a-47a4-98af-ed05f10d9e30 None admin] RBAC: Authorizing `identity:get_project(project_id=debcfc4c17a34d8a80f13016124abfaf)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:08.767879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d15fc-b54a-47a4-98af-ed05f10d9e30 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:08.769846 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 192/375] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:27:08 2022] GET /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:08.776147 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d15fc-b54a-47a4-98af-ed05f10d9e30 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:08.779986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:08.821631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb8e65-98ab-40a9-adf2-fc1a9fa72062 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:08.884814 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acd226-d200-4953-8e01-bb21f9c4cf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:08.885745 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acd226-d200-4953-8e01-bb21f9c4cf60 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:08.886038 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acd226-d200-4953-8e01-bb21f9c4cf60 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:08.886319 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acd226-d200-4953-8e01-bb21f9c4cf60 None admin] PATH_INFO: `/v3/projects/3f0d9b0cc6e44015a1eedda4627f582b` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:08.889245 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7acd226-d200-4953-8e01-bb21f9c4cf60 None admin] RBAC: Authorizing `identity:get_project(project_id=3f0d9b0cc6e44015a1eedda4627f582b)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:08.893290 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7acd226-d200-4953-8e01-bb21f9c4cf60 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:08.896292 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 184/376] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:27:08 2022] GET /identity/v3/projects/3f0d9b0cc6e44015a1eedda4627f582b => generated 344 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:08.900537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acd226-d200-4953-8e01-bb21f9c4cf60 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:08.940403 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d15fc-b54a-47a4-98af-ed05f10d9e30 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:09.006274 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b6fb5-1773-420a-9529-b905a3658192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:09.007209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b6fb5-1773-420a-9529-b905a3658192 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:09.007497 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b6fb5-1773-420a-9529-b905a3658192 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:09.007778 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b6fb5-1773-420a-9529-b905a3658192 None admin] PATH_INFO: `/v3/groups/fe95c2a6f34746a8b469bf15af149d71` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:09.009434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b6fb5-1773-420a-9529-b905a3658192 None admin] RBAC: Authorizing `identity:get_group(group_id=fe95c2a6f34746a8b469bf15af149d71)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:09.010907 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b6fb5-1773-420a-9529-b905a3658192 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:09.012562 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 193/377] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 22:27:08 2022] GET /identity/v3/groups/fe95c2a6f34746a8b469bf15af149d71 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:09.021091 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b6fb5-1773-420a-9529-b905a3658192 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:09.061910 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acd226-d200-4953-8e01-bb21f9c4cf60 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:09.099930 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611a07d-1272-4412-a242-409543dfb432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:09.101194 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611a07d-1272-4412-a242-409543dfb432 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:09.101779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611a07d-1272-4412-a242-409543dfb432 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:09.102801 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611a07d-1272-4412-a242-409543dfb432 None admin] PATH_INFO: `/v3/groups/ddc2bb71b27848ebace38b23499197e4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:09.108201 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611a07d-1272-4412-a242-409543dfb432 None admin] RBAC: Authorizing `identity:get_group(group_id=ddc2bb71b27848ebace38b23499197e4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:09.109983 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611a07d-1272-4412-a242-409543dfb432 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:09.112162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 185/378] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 22:27:08 2022] GET /identity/v3/groups/ddc2bb71b27848ebace38b23499197e4 => generated 229 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:09.121834 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611a07d-1272-4412-a242-409543dfb432 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:09.144964 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:09.161837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:09.219247 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b6fb5-1773-420a-9529-b905a3658192 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:09.257372 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdca37d-b261-4816-acc6-cc675d898c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:09.258955 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdca37d-b261-4816-acc6-cc675d898c56 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:09.259512 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdca37d-b261-4816-acc6-cc675d898c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:09.260020 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdca37d-b261-4816-acc6-cc675d898c56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:09.261240 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdca37d-b261-4816-acc6-cc675d898c56 None admin] RBAC: Adding query filter params (group.id=fe95c2a6f34746a8b469bf15af149d71, role.id=c3ad3cf6be504e35aaec10b7b9f3f34d, scope.project.id=debcfc4c17a34d8a80f13016124abfaf) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:09.261760 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdca37d-b261-4816-acc6-cc675d898c56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:09.263426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdca37d-b261-4816-acc6-cc675d898c56 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:09.273298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 194/379] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 23 22:27:09 2022] GET /identity/v3/role_assignments?group.id=fe95c2a6f34746a8b469bf15af149d71&scope.project.id=debcfc4c17a34d8a80f13016124abfaf&role.id=c3ad3cf6be504e35aaec10b7b9f3f34d => generated 260 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:09.355267 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611a07d-1272-4412-a242-409543dfb432 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:09.405667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986af49-4ebc-4748-9133-55970e8e57ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:09.406337 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986af49-4ebc-4748-9133-55970e8e57ed None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:09.406625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986af49-4ebc-4748-9133-55970e8e57ed None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:09.406903 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986af49-4ebc-4748-9133-55970e8e57ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:09.407745 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986af49-4ebc-4748-9133-55970e8e57ed None admin] RBAC: Adding query filter params (group.id=ddc2bb71b27848ebace38b23499197e4, role.id=6ff7708b59984b9287cc81d46a57b4b4, scope.project.id=3f0d9b0cc6e44015a1eedda4627f582b) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:09.408065 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986af49-4ebc-4748-9133-55970e8e57ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:09.409529 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986af49-4ebc-4748-9133-55970e8e57ed None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:09.416369 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 186/380] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 23 22:27:09 2022] GET /identity/v3/role_assignments?group.id=ddc2bb71b27848ebace38b23499197e4&scope.project.id=3f0d9b0cc6e44015a1eedda4627f582b&role.id=6ff7708b59984b9287cc81d46a57b4b4 => generated 260 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:09.729637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802913d-9e55-4606-aace-671370a517c5 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:09.730149 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802913d-9e55-4606-aace-671370a517c5 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:09.730578 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802913d-9e55-4606-aace-671370a517c5 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:09.732336 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 195/381] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:09.737946 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d87a8-4196-453e-a43f-047255df8650 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:09.738489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d87a8-4196-453e-a43f-047255df8650 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:09.738968 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d87a8-4196-453e-a43f-047255df8650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:09.761326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-1c2d87a8-4196-453e-a43f-047255df8650 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:09.762931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-1c2d87a8-4196-453e-a43f-047255df8650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:09.860018 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 187/382] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:09.903875 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802913d-9e55-4606-aace-671370a517c5 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:10.042343 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802913d-9e55-4606-aace-671370a517c5 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:10.091138 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195a841-a795-404b-ad05-5677ce34b1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:10.091958 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195a841-a795-404b-ad05-5677ce34b1cc None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:10.092244 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195a841-a795-404b-ad05-5677ce34b1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:10.092592 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195a841-a795-404b-ad05-5677ce34b1cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:10.097202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:10.107400 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195a841-a795-404b-ad05-5677ce34b1cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:10.115713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-f195a841-a795-404b-ad05-5677ce34b1cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:27:10.115713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:10.115713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:10.115713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:10.115713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:10.115713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:10.115713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:10.115713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:10.115713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:10.115713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:10.115713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:10.115713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:10.115713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:10.115713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:10.115713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:10.115713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:10.125527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:10.125527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:10.125527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:10.125527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:10.125527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:10.125527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:10.125527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:10.125527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:10.125527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:10.125527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:10.125527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:10.125527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:10.125527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:10.125527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:10.125527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:10.125527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:10.126340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:10.126340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:10.126340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:10.126340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:10.126340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:10.126340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:10.126340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:10.126340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:10.126340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:10.126340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:10.126340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:27:10.126340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:10.126340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 196/383] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:27:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:10.126997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d87a8-4196-453e-a43f-047255df8650 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:10.138088 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:10.271044 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d87a8-4196-453e-a43f-047255df8650 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:10.288501 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21633262-0633-48ee-b90d-5a696890d239 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:10.288833 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21633262-0633-48ee-b90d-5a696890d239 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:10.289120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21633262-0633-48ee-b90d-5a696890d239 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:10.291361 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 197/384] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:10.306300 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bedd51-c401-448a-8392-2359b9cd18c9 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:10.306549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bedd51-c401-448a-8392-2359b9cd18c9 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:10.306805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bedd51-c401-448a-8392-2359b9cd18c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:10.333690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-70bedd51-c401-448a-8392-2359b9cd18c9 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:10.334903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-70bedd51-c401-448a-8392-2359b9cd18c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:10.340535 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932230b-1f83-427a-9afe-a4c250351bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:10.341240 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932230b-1f83-427a-9afe-a4c250351bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:10.341528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932230b-1f83-427a-9afe-a4c250351bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:10.341807 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932230b-1f83-427a-9afe-a4c250351bd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:10.342566 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932230b-1f83-427a-9afe-a4c250351bd4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:10.342878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932230b-1f83-427a-9afe-a4c250351bd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:10.344254 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932230b-1f83-427a-9afe-a4c250351bd4 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:10.361081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 188/385] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:10.366640 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932230b-1f83-427a-9afe-a4c250351bd4 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:10.453944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 198/386] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:27:10.460017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:10.498277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932230b-1f83-427a-9afe-a4c250351bd4 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:10.505651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bedd51-c401-448a-8392-2359b9cd18c9 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:10.533766 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36283520-af0e-409b-b4c1-89759ea43fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:10.534991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36283520-af0e-409b-b4c1-89759ea43fdc None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:10.535487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36283520-af0e-409b-b4c1-89759ea43fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:10.535972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36283520-af0e-409b-b4c1-89759ea43fdc None admin] PATH_INFO: `/v3/groups/fe95c2a6f34746a8b469bf15af149d71` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:10.537416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36283520-af0e-409b-b4c1-89759ea43fdc None admin] RBAC: Authorizing `identity:get_group(group_id=fe95c2a6f34746a8b469bf15af149d71)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:10.539066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36283520-af0e-409b-b4c1-89759ea43fdc None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:10.541837 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 189/387] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 22:27:10 2022] GET /identity/v3/groups/fe95c2a6f34746a8b469bf15af149d71 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:27:10.549587 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36283520-af0e-409b-b4c1-89759ea43fdc None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:10.647498 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bedd51-c401-448a-8392-2359b9cd18c9 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:10.675486 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:10.685063 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38d70c-9b7b-49ae-b440-2a64b027012f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:10.692350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38d70c-9b7b-49ae-b440-2a64b027012f None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:10.692350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38d70c-9b7b-49ae-b440-2a64b027012f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:10.692350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38d70c-9b7b-49ae-b440-2a64b027012f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:10.710947 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38d70c-9b7b-49ae-b440-2a64b027012f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:10.718987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-8e38d70c-9b7b-49ae-b440-2a64b027012f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:27:10.718987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:10.718987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:10.718987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:10.718987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:10.718987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:10.718987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:10.718987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:10.718987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:10.718987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:10.718987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:10.718987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:10.718987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:10.718987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:10.718987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:10.718987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:10.720044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:10.720044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:10.720044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:10.720044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:10.720044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:10.720044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:10.720044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:10.720044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:10.720044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:10.720044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:10.720044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:10.720044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:10.720044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:10.720044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:10.720044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:10.720044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:10.720920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:10.720920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:10.720920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:10.720920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:10.720920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:10.720920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:10.720920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:10.720920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:10.720920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:10.720920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:10.720920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:27:10.720920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:10.720920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 199/388] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:27:10.728772 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38d70c-9b7b-49ae-b440-2a64b027012f None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:10.781770 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36283520-af0e-409b-b4c1-89759ea43fdc None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:10.838828 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38d70c-9b7b-49ae-b440-2a64b027012f None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:10.874629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab92e1-ee4b-429c-a860-21aef5f2eef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:10.875318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab92e1-ee4b-429c-a860-21aef5f2eef9 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:10.875613 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab92e1-ee4b-429c-a860-21aef5f2eef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:10.875732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30897c9-6852-4a44-9f04-bd36ae8c106c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:10.875917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab92e1-ee4b-429c-a860-21aef5f2eef9 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:10.876640 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30897c9-6852-4a44-9f04-bd36ae8c106c None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:10.876755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab92e1-ee4b-429c-a860-21aef5f2eef9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:10.876936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30897c9-6852-4a44-9f04-bd36ae8c106c None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:10.877091 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab92e1-ee4b-429c-a860-21aef5f2eef9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:10.877247 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30897c9-6852-4a44-9f04-bd36ae8c106c None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:10.878393 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab92e1-ee4b-429c-a860-21aef5f2eef9 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:10.881304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30897c9-6852-4a44-9f04-bd36ae8c106c None admin] RBAC: Authorizing `identity:get_project(project_id=24e47f2dd0c74c218f4fd2059483f785)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:10.882749 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30897c9-6852-4a44-9f04-bd36ae8c106c None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:10.884662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 190/389] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:27:10 2022] GET /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785 => generated 298 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:10.890875 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 200/390] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:27:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:10.893630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30897c9-6852-4a44-9f04-bd36ae8c106c None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:10.897655 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab92e1-ee4b-429c-a860-21aef5f2eef9 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:10.912965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:10.988992 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:11.026351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab92e1-ee4b-429c-a860-21aef5f2eef9 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:11.029346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30897c9-6852-4a44-9f04-bd36ae8c106c None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:11.060161 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d345f-1845-4850-8c6b-ef0a5f69a2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:11.061073 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d345f-1845-4850-8c6b-ef0a5f69a2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:11.061355 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d345f-1845-4850-8c6b-ef0a5f69a2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:11.061626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d345f-1845-4850-8c6b-ef0a5f69a2d7 None admin] PATH_INFO: `/v3/groups/fe95c2a6f34746a8b469bf15af149d71` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:11.062805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d345f-1845-4850-8c6b-ef0a5f69a2d7 None admin] RBAC: Authorizing `identity:get_group(group_id=fe95c2a6f34746a8b469bf15af149d71)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:11.064361 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d345f-1845-4850-8c6b-ef0a5f69a2d7 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:11.066247 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c07aa0-5ca2-432b-883e-b58a5b27f40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:11.066555 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 201/391] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 22:27:10 2022] GET /identity/v3/groups/fe95c2a6f34746a8b469bf15af149d71 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:11.067135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c07aa0-5ca2-432b-883e-b58a5b27f40f None admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:11.067431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c07aa0-5ca2-432b-883e-b58a5b27f40f None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:11.067711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c07aa0-5ca2-432b-883e-b58a5b27f40f None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785/groups/fe95c2a6f34746a8b469bf15af149d71/roles/c3ad3cf6be504e35aaec10b7b9f3f34d` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:11.067818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0fb6a-e41e-4422-83b7-ad25b873d238 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:11.067993 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0fb6a-e41e-4422-83b7-ad25b873d238 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:11.068271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0fb6a-e41e-4422-83b7-ad25b873d238 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:11.071013 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c07aa0-5ca2-432b-883e-b58a5b27f40f None admin] RBAC: Authorizing `identity:create_grant(project_id=24e47f2dd0c74c218f4fd2059483f785, group_id=fe95c2a6f34746a8b469bf15af149d71, role_id=c3ad3cf6be504e35aaec10b7b9f3f34d)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:11.072674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c07aa0-5ca2-432b-883e-b58a5b27f40f None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:11.077942 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 202/392] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:11 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:11.078757 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0fb6a-e41e-4422-83b7-ad25b873d238 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:11.151410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 191/393] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Jun 23 22:27:10 2022] PUT /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785/groups/fe95c2a6f34746a8b469bf15af149d71/roles/c3ad3cf6be504e35aaec10b7b9f3f34d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:27:11.152711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a7385-7ac5-4344-8ee5-09bf3fce588c None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:11.153245 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a7385-7ac5-4344-8ee5-09bf3fce588c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:11.153538 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a7385-7ac5-4344-8ee5-09bf3fce588c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:11.186009 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0fb6a-e41e-4422-83b7-ad25b873d238 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:11.187816 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-ee6a7385-7ac5-4344-8ee5-09bf3fce588c None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:11.192868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ee6a7385-7ac5-4344-8ee5-09bf3fce588c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:11.208651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:11.243556 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa4f31-5843-416a-b816-2e71177965ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:11.244432 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa4f31-5843-416a-b816-2e71177965ab None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:11.244737 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa4f31-5843-416a-b816-2e71177965ab None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:11.245021 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa4f31-5843-416a-b816-2e71177965ab None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:11.246226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa4f31-5843-416a-b816-2e71177965ab None admin] RBAC: Authorizing `identity:get_project(project_id=24e47f2dd0c74c218f4fd2059483f785)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:11.247664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa4f31-5843-416a-b816-2e71177965ab None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:11.252726 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 203/394] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:27:11 2022] GET /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:11.255181 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa4f31-5843-416a-b816-2e71177965ab None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:11.275375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:11.289167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 192/395] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:11.318439 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a7385-7ac5-4344-8ee5-09bf3fce588c None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:11.436949 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a7385-7ac5-4344-8ee5-09bf3fce588c None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:11.442955 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa4f31-5843-416a-b816-2e71177965ab None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:11.478018 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e883ed-5046-4f94-9acb-66b072735b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:11.479424 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e883ed-5046-4f94-9acb-66b072735b18 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:11.479873 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e883ed-5046-4f94-9acb-66b072735b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:11.480324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e883ed-5046-4f94-9acb-66b072735b18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:11.482559 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be9ea8-91b1-4368-a3be-5bed3188e1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:11.483553 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be9ea8-91b1-4368-a3be-5bed3188e1f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:11.483839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be9ea8-91b1-4368-a3be-5bed3188e1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:11.484115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be9ea8-91b1-4368-a3be-5bed3188e1f1 None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785/groups/fe95c2a6f34746a8b469bf15af149d71/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:11.486234 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5be9ea8-91b1-4368-a3be-5bed3188e1f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=24e47f2dd0c74c218f4fd2059483f785, group_id=fe95c2a6f34746a8b469bf15af149d71, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:11.487664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5be9ea8-91b1-4368-a3be-5bed3188e1f1 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:11.494566 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e883ed-5046-4f94-9acb-66b072735b18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:11.503596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-97e883ed-5046-4f94-9acb-66b072735b18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:27:11.503596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:11.503596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:11.503596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:11.503596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:11.503596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:11.503596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:11.503596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:11.503596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:11.503596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:11.503596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:11.503596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:11.503596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:11.503596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:11.503596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:11.503596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:11.504855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:11.504855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:11.504855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:11.504855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:11.504855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:11.504855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:11.504855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:11.504855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:11.504855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:11.504855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:11.504855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:11.504855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:11.504855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:11.504855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:11.504855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:11.504855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:11.505859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:11.505859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:11.505859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:11.505859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:11.505859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:11.505859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:11.505859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:11.505859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:11.505859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:11.505859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:11.505859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:27:11.505859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:11.505859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 193/396] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:11.511308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e883ed-5046-4f94-9acb-66b072735b18 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:11.554724 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 204/397] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Jun 23 22:27:11 2022] PUT /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785/groups/fe95c2a6f34746a8b469bf15af149d71/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:27:11.581204 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:11.639173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e883ed-5046-4f94-9acb-66b072735b18 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:11.686432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e530f58-9271-4296-984a-cdd8d3e6c46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:11.687178 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e530f58-9271-4296-984a-cdd8d3e6c46f None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:11.687410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e530f58-9271-4296-984a-cdd8d3e6c46f None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:11.687696 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e530f58-9271-4296-984a-cdd8d3e6c46f None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:11.688532 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e530f58-9271-4296-984a-cdd8d3e6c46f None admin] RBAC: Adding query filter params (name=member) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:11.688877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e530f58-9271-4296-984a-cdd8d3e6c46f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:11.690400 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e530f58-9271-4296-984a-cdd8d3e6c46f None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:11.703747 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 194/398] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:27:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:27:11.709552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be9ea8-91b1-4368-a3be-5bed3188e1f1 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:11.770791 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:11.846973 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be9ea8-91b1-4368-a3be-5bed3188e1f1 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:11.876444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87c79c-e43e-4653-9f43-2916581a6fb7 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:11.877124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87c79c-e43e-4653-9f43-2916581a6fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:11.877601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87c79c-e43e-4653-9f43-2916581a6fb7 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:11.879284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 195/399] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:11.886455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27283f-f54b-4dea-ae17-fbfa984aeb41 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:11.886749 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27283f-f54b-4dea-ae17-fbfa984aeb41 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:11.886916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279b36c-dc40-4e1d-a58d-66c95701151d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:11.887036 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27283f-f54b-4dea-ae17-fbfa984aeb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:11.887793 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279b36c-dc40-4e1d-a58d-66c95701151d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:11.888081 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279b36c-dc40-4e1d-a58d-66c95701151d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:11.888363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279b36c-dc40-4e1d-a58d-66c95701151d None admin] PATH_INFO: `/v3/groups/fe95c2a6f34746a8b469bf15af149d71` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:11.889618 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279b36c-dc40-4e1d-a58d-66c95701151d None admin] RBAC: Authorizing `identity:get_group(group_id=fe95c2a6f34746a8b469bf15af149d71)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:11.891111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279b36c-dc40-4e1d-a58d-66c95701151d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:11.893045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 205/400] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 22:27:11 2022] GET /identity/v3/groups/fe95c2a6f34746a8b469bf15af149d71 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:11.898429 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279b36c-dc40-4e1d-a58d-66c95701151d None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:11.910478 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-dc27283f-f54b-4dea-ae17-fbfa984aeb41 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:11.911690 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-dc27283f-f54b-4dea-ae17-fbfa984aeb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:12.017936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 196/401] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:27:12.024541 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91553490-acce-4cda-9199-557c159323c5 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:12.024541 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91553490-acce-4cda-9199-557c159323c5 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:12.024541 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91553490-acce-4cda-9199-557c159323c5 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:12.024541 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 197/402] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:12.025886 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279b36c-dc40-4e1d-a58d-66c95701151d None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:12.041023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d36658-94ea-4e50-af19-a781ea014d41 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:12.041316 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d36658-94ea-4e50-af19-a781ea014d41 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:12.041600 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d36658-94ea-4e50-af19-a781ea014d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:12.049287 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:12.061567 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-52d36658-94ea-4e50-af19-a781ea014d41 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:12.068494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-52d36658-94ea-4e50-af19-a781ea014d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:12.074034 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ffd76-23b0-4293-989e-82d9e1824b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:12.075555 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ffd76-23b0-4293-989e-82d9e1824b58 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:12.076142 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ffd76-23b0-4293-989e-82d9e1824b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:12.076674 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ffd76-23b0-4293-989e-82d9e1824b58 None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:12.078335 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ffd76-23b0-4293-989e-82d9e1824b58 None admin] RBAC: Authorizing `identity:get_project(project_id=debcfc4c17a34d8a80f13016124abfaf)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:12.080128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ffd76-23b0-4293-989e-82d9e1824b58 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:12.082526 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 206/403] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:27:11 2022] GET /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:12.088427 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ffd76-23b0-4293-989e-82d9e1824b58 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:12.114337 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:12.154210 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 198/404] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:27:12.154210 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d36658-94ea-4e50-af19-a781ea014d41 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:12.234801 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ffd76-23b0-4293-989e-82d9e1824b58 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:12.278026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c8778d-20ad-45a0-bcf8-d1fbbd5b38b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:12.278993 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8778d-20ad-45a0-bcf8-d1fbbd5b38b7 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:12.279170 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8778d-20ad-45a0-bcf8-d1fbbd5b38b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:12.279414 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8778d-20ad-45a0-bcf8-d1fbbd5b38b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:12.282683 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d36658-94ea-4e50-af19-a781ea014d41 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:12.296702 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c8778d-20ad-45a0-bcf8-d1fbbd5b38b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:12.304864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-38c8778d-20ad-45a0-bcf8-d1fbbd5b38b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:27:12.304864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:12.304864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:12.304864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:12.304864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:12.304864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:12.304864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:12.304864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:12.304864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:12.304864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:12.304864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:12.304864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:12.304864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:12.304864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:12.304864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:12.304864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:12.306342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:12.306342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:12.306342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:12.306342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:12.306342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:12.306342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:12.306342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:12.306342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:12.306342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:12.306342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:12.306342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:12.306342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:12.306342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:12.306342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:12.306342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:12.306342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:12.307137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:12.307137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:12.307137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:12.307137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:12.307137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:12.307137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:12.307137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:12.307137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:12.307137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:12.307137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:12.307137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:27:12.307137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:12.307137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 207/405] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:27:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:12.307804 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c8778d-20ad-45a0-bcf8-d1fbbd5b38b7 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:12.338380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c14486-06d1-4316-bbfa-8c0ef40db6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:12.340073 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c14486-06d1-4316-bbfa-8c0ef40db6d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:12.340851 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c14486-06d1-4316-bbfa-8c0ef40db6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:12.341330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c14486-06d1-4316-bbfa-8c0ef40db6d5 None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf/groups/fe95c2a6f34746a8b469bf15af149d71/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:12.344571 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c14486-06d1-4316-bbfa-8c0ef40db6d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=debcfc4c17a34d8a80f13016124abfaf, group_id=fe95c2a6f34746a8b469bf15af149d71, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:12.346826 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c14486-06d1-4316-bbfa-8c0ef40db6d5 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:12.358446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:12.417336 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c8778d-20ad-45a0-bcf8-d1fbbd5b38b7 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:12.451797 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0dbf9-b69a-4f42-95cf-04115ff73b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:12.452636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0dbf9-b69a-4f42-95cf-04115ff73b62 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:12.452926 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0dbf9-b69a-4f42-95cf-04115ff73b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:12.453227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0dbf9-b69a-4f42-95cf-04115ff73b62 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:12.464100 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0dbf9-b69a-4f42-95cf-04115ff73b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:12.472189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-09a0dbf9-b69a-4f42-95cf-04115ff73b62 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:27:12.472189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:12.472189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:12.472189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:12.472189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:12.472189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:12.472189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:12.472189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:12.472189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:12.472189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:12.472189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:12.472189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:12.472189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:12.472189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:12.472189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:12.472189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:12.474128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:12.474128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:12.474128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:12.474128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:12.474128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:12.474128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:12.474128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:12.474128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:12.474128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:12.474128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:12.474128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:12.474128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:12.474128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:12.474128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:12.474128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:12.474128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:12.474912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:12.474912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:12.474912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:12.474912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:12.474912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:12.474912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:12.474912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:12.474912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:12.474912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:12.474912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:12.474912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:27:12.474912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:12.474912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 208/406] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:27:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:12.474912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0dbf9-b69a-4f42-95cf-04115ff73b62 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:12.504695 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 199/407] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Jun 23 22:27:12 2022] PUT /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf/groups/fe95c2a6f34746a8b469bf15af149d71/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:27:12.506083 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c14486-06d1-4316-bbfa-8c0ef40db6d5 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:12.591580 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:12.692070 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:12.695141 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0dbf9-b69a-4f42-95cf-04115ff73b62 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:12.708316 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c14486-06d1-4316-bbfa-8c0ef40db6d5 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:12.736182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f0c90-484d-4f34-86e9-2042ff8e1d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:12.737318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f0c90-484d-4f34-86e9-2042ff8e1d74 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:12.737851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f0c90-484d-4f34-86e9-2042ff8e1d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:12.738368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f0c90-484d-4f34-86e9-2042ff8e1d74 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:12.739328 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f0c90-484d-4f34-86e9-2042ff8e1d74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:12.739928 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f0c90-484d-4f34-86e9-2042ff8e1d74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:12.741637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f0c90-484d-4f34-86e9-2042ff8e1d74 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:12.747040 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012f576-b9e9-43ba-9378-5bb50ce91a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:12.747719 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012f576-b9e9-43ba-9378-5bb50ce91a45 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:12.748004 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012f576-b9e9-43ba-9378-5bb50ce91a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:12.748283 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012f576-b9e9-43ba-9378-5bb50ce91a45 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:12.749281 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012f576-b9e9-43ba-9378-5bb50ce91a45 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:12.749457 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012f576-b9e9-43ba-9378-5bb50ce91a45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:12.750909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012f576-b9e9-43ba-9378-5bb50ce91a45 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:12.757931 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 209/408] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:12.760387 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 200/409] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:12.761097 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f0c90-484d-4f34-86e9-2042ff8e1d74 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:12.768618 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012f576-b9e9-43ba-9378-5bb50ce91a45 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:12.885051 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f0c90-484d-4f34-86e9-2042ff8e1d74 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:12.921769 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb509eb-9bca-4b0a-96f5-3d10fba0d396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:12.923010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb509eb-9bca-4b0a-96f5-3d10fba0d396 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:12.923517 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb509eb-9bca-4b0a-96f5-3d10fba0d396 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:12.923992 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb509eb-9bca-4b0a-96f5-3d10fba0d396 None admin] PATH_INFO: `/v3/groups/ddc2bb71b27848ebace38b23499197e4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:12.925607 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb509eb-9bca-4b0a-96f5-3d10fba0d396 None admin] RBAC: Authorizing `identity:get_group(group_id=ddc2bb71b27848ebace38b23499197e4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:12.927488 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb509eb-9bca-4b0a-96f5-3d10fba0d396 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:12.929880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 210/410] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 22:27:12 2022] GET /identity/v3/groups/ddc2bb71b27848ebace38b23499197e4 => generated 229 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:12.937734 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb509eb-9bca-4b0a-96f5-3d10fba0d396 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:12.984985 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:12.987375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012f576-b9e9-43ba-9378-5bb50ce91a45 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:13.004960 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:13.056698 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c7c9f-73f1-4556-b1ab-30d1568d5a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:13.057577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c7c9f-73f1-4556-b1ab-30d1568d5a4d None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:13.057928 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c7c9f-73f1-4556-b1ab-30d1568d5a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:13.061044 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c7c9f-73f1-4556-b1ab-30d1568d5a4d None admin] PATH_INFO: `/v3/groups/fe95c2a6f34746a8b469bf15af149d71` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:13.063254 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c7c9f-73f1-4556-b1ab-30d1568d5a4d None admin] RBAC: Authorizing `identity:get_group(group_id=fe95c2a6f34746a8b469bf15af149d71)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:13.064737 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c7c9f-73f1-4556-b1ab-30d1568d5a4d None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:13.075039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 201/411] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 22:27:12 2022] GET /identity/v3/groups/fe95c2a6f34746a8b469bf15af149d71 => generated 226 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:27:13.087564 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c7c9f-73f1-4556-b1ab-30d1568d5a4d None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:13.170671 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb509eb-9bca-4b0a-96f5-3d10fba0d396 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:13.203581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c7c9f-73f1-4556-b1ab-30d1568d5a4d None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:13.240894 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405e78b-6e57-4c2e-ae25-597e2eaaa1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:13.241764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405e78b-6e57-4c2e-ae25-597e2eaaa1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:13.243634 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405e78b-6e57-4c2e-ae25-597e2eaaa1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:13.243918 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405e78b-6e57-4c2e-ae25-597e2eaaa1d5 None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:13.246388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405e78b-6e57-4c2e-ae25-597e2eaaa1d5 None admin] RBAC: Authorizing `identity:get_project(project_id=debcfc4c17a34d8a80f13016124abfaf)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:13.247838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405e78b-6e57-4c2e-ae25-597e2eaaa1d5 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:13.252756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 202/412] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:27:13 2022] GET /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:27:13.266674 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582145e7-b347-4922-986d-860ad18e6fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:13.267552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582145e7-b347-4922-986d-860ad18e6fbd None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:13.267835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582145e7-b347-4922-986d-860ad18e6fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:13.268113 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582145e7-b347-4922-986d-860ad18e6fbd None admin] PATH_INFO: `/v3/projects/3f0d9b0cc6e44015a1eedda4627f582b` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:13.269391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582145e7-b347-4922-986d-860ad18e6fbd None admin] RBAC: Authorizing `identity:get_project(project_id=3f0d9b0cc6e44015a1eedda4627f582b)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:13.270883 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582145e7-b347-4922-986d-860ad18e6fbd None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:13.279025 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405e78b-6e57-4c2e-ae25-597e2eaaa1d5 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:13.281747 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 211/413] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:27:12 2022] GET /identity/v3/projects/3f0d9b0cc6e44015a1eedda4627f582b => generated 344 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:13.287914 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582145e7-b347-4922-986d-860ad18e6fbd None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:13.329301 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:13.406387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:13.410324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582145e7-b347-4922-986d-860ad18e6fbd None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:13.447120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c476514-d0d4-439a-9fc5-968788ba4192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:13.447939 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c476514-d0d4-439a-9fc5-968788ba4192 None admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:13.448221 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c476514-d0d4-439a-9fc5-968788ba4192 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:13.448496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c476514-d0d4-439a-9fc5-968788ba4192 None admin] PATH_INFO: `/v3/projects/3f0d9b0cc6e44015a1eedda4627f582b/groups/ddc2bb71b27848ebace38b23499197e4/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:13.450496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c476514-d0d4-439a-9fc5-968788ba4192 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f0d9b0cc6e44015a1eedda4627f582b, group_id=ddc2bb71b27848ebace38b23499197e4, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:13.451944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c476514-d0d4-439a-9fc5-968788ba4192 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:13.453007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405e78b-6e57-4c2e-ae25-597e2eaaa1d5 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:13.504685 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d25ec-e237-439a-9fdc-67efe58e7497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:13.505583 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d25ec-e237-439a-9fdc-67efe58e7497 None admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:13.505876 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d25ec-e237-439a-9fdc-67efe58e7497 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:13.506161 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d25ec-e237-439a-9fdc-67efe58e7497 None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf/groups/fe95c2a6f34746a8b469bf15af149d71/roles/c3ad3cf6be504e35aaec10b7b9f3f34d` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:13.508988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d25ec-e237-439a-9fdc-67efe58e7497 None admin] RBAC: Authorizing `identity:create_grant(project_id=debcfc4c17a34d8a80f13016124abfaf, group_id=fe95c2a6f34746a8b469bf15af149d71, role_id=c3ad3cf6be504e35aaec10b7b9f3f34d)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:13.510598 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d25ec-e237-439a-9fdc-67efe58e7497 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:13.528930 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 212/414] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Jun 23 22:27:13 2022] PUT /identity/v3/projects/3f0d9b0cc6e44015a1eedda4627f582b/groups/ddc2bb71b27848ebace38b23499197e4/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:27:13.550091 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b50270-83d6-43aa-a6d1-6b0f485ff8c9 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:13.550521 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b50270-83d6-43aa-a6d1-6b0f485ff8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:13.550904 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b50270-83d6-43aa-a6d1-6b0f485ff8c9 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:13.556816 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 213/415] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:27:13.568065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680a911-9f56-4c72-a872-6a794190eaa0 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:13.568065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680a911-9f56-4c72-a872-6a794190eaa0 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:13.568065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680a911-9f56-4c72-a872-6a794190eaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:13.578692 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-e680a911-9f56-4c72-a872-6a794190eaa0 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:13.580052 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e680a911-9f56-4c72-a872-6a794190eaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:13.603853 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 203/416] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Jun 23 22:27:13 2022] PUT /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf/groups/fe95c2a6f34746a8b469bf15af149d71/roles/c3ad3cf6be504e35aaec10b7b9f3f34d => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:27:13.687041 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 214/417] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:27:13.730842 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d25ec-e237-439a-9fdc-67efe58e7497 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:13.836418 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:13.846574 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d25ec-e237-439a-9fdc-67efe58e7497 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:13.888558 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033109a-a08e-4b88-9c9e-474ef51d7327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:13.889415 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033109a-a08e-4b88-9c9e-474ef51d7327 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:13.889705 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033109a-a08e-4b88-9c9e-474ef51d7327 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:13.889989 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033109a-a08e-4b88-9c9e-474ef51d7327 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:13.902089 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033109a-a08e-4b88-9c9e-474ef51d7327 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:13.910280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-c033109a-a08e-4b88-9c9e-474ef51d7327 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:27:13.910280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:13.910280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:13.910280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:13.910280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:13.910280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:13.910280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:13.910280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:13.910280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:13.910280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:13.910280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:13.910280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:13.910280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:13.910280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:13.910280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:13.910280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:13.911147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:13.911147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:13.911147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:13.911147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:13.911147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:13.911147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:13.911147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:13.911147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:13.911147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:13.911147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:13.911147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:13.911147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:13.911147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:13.911147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:13.911147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:13.911147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:13.912243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:13.912243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:13.912243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:13.912243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:13.912243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:13.912243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:13.912243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:13.912243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:13.912243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:13.912243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:13.912243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:27:13.912243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:13.912243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 204/418] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:27:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:13.921613 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680a911-9f56-4c72-a872-6a794190eaa0 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:13.977206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0c5c8-ab84-444b-b136-644fdbac71e0 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:13.977705 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0c5c8-ab84-444b-b136-644fdbac71e0 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:13.978149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0c5c8-ab84-444b-b136-644fdbac71e0 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:13.979681 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 205/419] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:13.984764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8200b0-c914-4acc-a58d-7b532de19e72 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:13.985000 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8200b0-c914-4acc-a58d-7b532de19e72 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:13.985286 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8200b0-c914-4acc-a58d-7b532de19e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:14.017629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:14.018198 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-ce8200b0-c914-4acc-a58d-7b532de19e72 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:14.019413 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ce8200b0-c914-4acc-a58d-7b532de19e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:14.037833 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680a911-9f56-4c72-a872-6a794190eaa0 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:14.112034 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97e804-97ca-4a56-b4f0-15ed906c3486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:14.112836 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97e804-97ca-4a56-b4f0-15ed906c3486 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:14.113131 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97e804-97ca-4a56-b4f0-15ed906c3486 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:14.113410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97e804-97ca-4a56-b4f0-15ed906c3486 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:14.114174 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97e804-97ca-4a56-b4f0-15ed906c3486 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:14.114482 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97e804-97ca-4a56-b4f0-15ed906c3486 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:14.115877 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97e804-97ca-4a56-b4f0-15ed906c3486 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:14.126165 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 215/420] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:14.131594 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97e804-97ca-4a56-b4f0-15ed906c3486 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:14.167979 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 206/421] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:27:14.209369 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8200b0-c914-4acc-a58d-7b532de19e72 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:14.308020 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97e804-97ca-4a56-b4f0-15ed906c3486 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:14.361758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:14.380196 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ba0f5-7498-48ec-9b3a-3c984dae5b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:14.380874 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ba0f5-7498-48ec-9b3a-3c984dae5b29 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:14.381161 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ba0f5-7498-48ec-9b3a-3c984dae5b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:14.381437 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ba0f5-7498-48ec-9b3a-3c984dae5b29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:14.399549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8200b0-c914-4acc-a58d-7b532de19e72 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:14.416449 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:14.459986 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dbc2a-b6ba-4d48-b89f-a2c15b0cf07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:14.460810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dbc2a-b6ba-4d48-b89f-a2c15b0cf07f None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:14.461091 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dbc2a-b6ba-4d48-b89f-a2c15b0cf07f None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:14.461431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dbc2a-b6ba-4d48-b89f-a2c15b0cf07f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:14.482643 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ba0f5-7498-48ec-9b3a-3c984dae5b29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:14.484112 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ba0f5-7498-48ec-9b3a-3c984dae5b29 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:14.491792 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109dbc2a-b6ba-4d48-b89f-a2c15b0cf07f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:14.501124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-109dbc2a-b6ba-4d48-b89f-a2c15b0cf07f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:27:14.501124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:14.501124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:14.501124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:14.501124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:14.501124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:14.501124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:14.501124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:14.501124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:14.501124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:14.501124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:14.501124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:14.501124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:14.501124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:14.501124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:14.501124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:14.502035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:14.502035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:14.502035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:14.502035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:14.502035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:14.502035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:14.502035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:14.502035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:14.502035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:14.502035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:14.502035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:14.502035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:14.502035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:14.502035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:14.502035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:14.502035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:14.502824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:14.502824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:14.502824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:14.502824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:14.502824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:14.502824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:14.502824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:14.502824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:14.502824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:14.502824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:14.502824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:27:14.502824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:14.502824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 207/422] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:27:14.513605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dbc2a-b6ba-4d48-b89f-a2c15b0cf07f None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:14.594113 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 216/423] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:14.595272 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ba0f5-7498-48ec-9b3a-3c984dae5b29 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:14.660111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:14.661059 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dbc2a-b6ba-4d48-b89f-a2c15b0cf07f None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:14.696507 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a05b5-dfea-4ad0-b0b2-08a223304427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:14.697220 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a05b5-dfea-4ad0-b0b2-08a223304427 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:14.697505 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a05b5-dfea-4ad0-b0b2-08a223304427 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:14.697783 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a05b5-dfea-4ad0-b0b2-08a223304427 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:14.698525 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a05b5-dfea-4ad0-b0b2-08a223304427 None admin] RBAC: Adding query filter params (name=member) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:14.698831 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a05b5-dfea-4ad0-b0b2-08a223304427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:14.700221 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a05b5-dfea-4ad0-b0b2-08a223304427 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:14.708967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 208/424] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:27:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:14.710603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ba0f5-7498-48ec-9b3a-3c984dae5b29 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:14.715023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a05b5-dfea-4ad0-b0b2-08a223304427 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:14.718660 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:14.759301 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4768a35-5d26-40d7-abfb-92d96eb70330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:14.760255 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4768a35-5d26-40d7-abfb-92d96eb70330 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:14.760543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4768a35-5d26-40d7-abfb-92d96eb70330 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:14.760847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4768a35-5d26-40d7-abfb-92d96eb70330 None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:14.765328 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4768a35-5d26-40d7-abfb-92d96eb70330 None admin] RBAC: Authorizing `identity:get_project(project_id=24e47f2dd0c74c218f4fd2059483f785)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:14.766783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4768a35-5d26-40d7-abfb-92d96eb70330 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:14.768828 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 217/425] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:27:14 2022] GET /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:27:14.773872 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4768a35-5d26-40d7-abfb-92d96eb70330 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:14.834503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a05b5-dfea-4ad0-b0b2-08a223304427 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:14.870996 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60aa5c2-53a7-40c6-a4d9-c38e40f34d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:14.871568 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60aa5c2-53a7-40c6-a4d9-c38e40f34d06 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:14.871856 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60aa5c2-53a7-40c6-a4d9-c38e40f34d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:14.872152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60aa5c2-53a7-40c6-a4d9-c38e40f34d06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:14.943589 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60aa5c2-53a7-40c6-a4d9-c38e40f34d06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:14.945179 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60aa5c2-53a7-40c6-a4d9-c38e40f34d06 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:14.948799 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:14.973560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4768a35-5d26-40d7-abfb-92d96eb70330 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:15.012973 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:15.044952 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad4c11-6c09-4ec3-a991-8689c13feea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:15.045888 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad4c11-6c09-4ec3-a991-8689c13feea6 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:15.046185 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad4c11-6c09-4ec3-a991-8689c13feea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:15.046469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad4c11-6c09-4ec3-a991-8689c13feea6 None admin] PATH_INFO: `/v3/groups/fe95c2a6f34746a8b469bf15af149d71` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:15.053771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad4c11-6c09-4ec3-a991-8689c13feea6 None admin] RBAC: Authorizing `identity:get_group(group_id=fe95c2a6f34746a8b469bf15af149d71)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:15.055316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad4c11-6c09-4ec3-a991-8689c13feea6 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:15.056559 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 209/426] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 22:27:15.057355 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 218/427] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 22:27:14 2022] GET /identity/v3/groups/fe95c2a6f34746a8b469bf15af149d71 => generated 226 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:27:15.062724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60aa5c2-53a7-40c6-a4d9-c38e40f34d06 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:15.067202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad4c11-6c09-4ec3-a991-8689c13feea6 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:15.168383 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60aa5c2-53a7-40c6-a4d9-c38e40f34d06 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:15.170725 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:15.204307 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cf065-b333-4998-8e6d-eda0f56c1ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:15.205199 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cf065-b333-4998-8e6d-eda0f56c1ecc None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:15.205484 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cf065-b333-4998-8e6d-eda0f56c1ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:15.205760 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cf065-b333-4998-8e6d-eda0f56c1ecc None admin] PATH_INFO: `/v3/projects/24e47f2dd0c74c218f4fd2059483f785` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:15.207913 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cf065-b333-4998-8e6d-eda0f56c1ecc None admin] RBAC: Authorizing `identity:get_project(project_id=24e47f2dd0c74c218f4fd2059483f785)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:15.209396 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cf065-b333-4998-8e6d-eda0f56c1ecc None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:15.211395 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 210/428] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:27:15 2022] GET /identity/v3/projects/24e47f2dd0c74c218f4fd2059483f785 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:15.216664 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cf065-b333-4998-8e6d-eda0f56c1ecc None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:15.308688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad4c11-6c09-4ec3-a991-8689c13feea6 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:15.321601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cf065-b333-4998-8e6d-eda0f56c1ecc None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:15.345110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec3bd6-b463-4b61-9665-cf34e1c0b633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:15.346061 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec3bd6-b463-4b61-9665-cf34e1c0b633 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:15.346541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec3bd6-b463-4b61-9665-cf34e1c0b633 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:15.347039 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec3bd6-b463-4b61-9665-cf34e1c0b633 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:15.348061 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec3bd6-b463-4b61-9665-cf34e1c0b633 None admin] RBAC: Adding query filter params (group.id=fe95c2a6f34746a8b469bf15af149d71, role.id=c3ad3cf6be504e35aaec10b7b9f3f34d, scope.project.id=24e47f2dd0c74c218f4fd2059483f785) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:15.348570 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec3bd6-b463-4b61-9665-cf34e1c0b633 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:15.350261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec3bd6-b463-4b61-9665-cf34e1c0b633 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:15.358436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57dd05-6c95-4f15-91d8-4c97596fc9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:15.364607 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57dd05-6c95-4f15-91d8-4c97596fc9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:15.364607 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57dd05-6c95-4f15-91d8-4c97596fc9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:15.364607 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57dd05-6c95-4f15-91d8-4c97596fc9e1 None admin] PATH_INFO: `/v3/groups/fe95c2a6f34746a8b469bf15af149d71` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:15.364607 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57dd05-6c95-4f15-91d8-4c97596fc9e1 None admin] RBAC: Authorizing `identity:get_group(group_id=fe95c2a6f34746a8b469bf15af149d71)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:15.364607 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57dd05-6c95-4f15-91d8-4c97596fc9e1 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:15.365051 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 211/430] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 22:27:15 2022] GET /identity/v3/groups/fe95c2a6f34746a8b469bf15af149d71 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:27:15.365681 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 219/430] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 23 22:27:15 2022] GET /identity/v3/role_assignments?group.id=fe95c2a6f34746a8b469bf15af149d71&scope.project.id=24e47f2dd0c74c218f4fd2059483f785&role.id=c3ad3cf6be504e35aaec10b7b9f3f34d => generated 611 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:27:15.372013 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953aa66-5303-42d1-b304-84e6aff1dab0 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:15.372307 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953aa66-5303-42d1-b304-84e6aff1dab0 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:15.372607 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953aa66-5303-42d1-b304-84e6aff1dab0 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:15.374367 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 212/431] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:15.376201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec3bd6-b463-4b61-9665-cf34e1c0b633 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:15.395424 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31cfd0-1124-4a1b-8739-d005f16b9c18 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:15.395982 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31cfd0-1124-4a1b-8739-d005f16b9c18 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:15.396434 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31cfd0-1124-4a1b-8739-d005f16b9c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:15.433944 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-ae31cfd0-1124-4a1b-8739-d005f16b9c18 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:15.435089 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ae31cfd0-1124-4a1b-8739-d005f16b9c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:15.451805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:15.504788 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:15.537383 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec3bd6-b463-4b61-9665-cf34e1c0b633 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:15.571941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 213/432] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:15.576214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac88ff-8a98-46da-bf2d-a3e1f2c7d484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:15.576909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac88ff-8a98-46da-bf2d-a3e1f2c7d484 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:15.577219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac88ff-8a98-46da-bf2d-a3e1f2c7d484 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:15.577475 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac88ff-8a98-46da-bf2d-a3e1f2c7d484 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:15.578316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac88ff-8a98-46da-bf2d-a3e1f2c7d484 None admin] RBAC: Adding query filter params (group.id=fe95c2a6f34746a8b469bf15af149d71, role.id=1305747c745d49dc9a9718b7aa8aa4a9, scope.project.id=24e47f2dd0c74c218f4fd2059483f785) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:15.578629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac88ff-8a98-46da-bf2d-a3e1f2c7d484 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:15.580080 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac88ff-8a98-46da-bf2d-a3e1f2c7d484 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:15.587067 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 220/433] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 23 22:27:15 2022] GET /identity/v3/role_assignments?group.id=fe95c2a6f34746a8b469bf15af149d71&scope.project.id=24e47f2dd0c74c218f4fd2059483f785&role.id=1305747c745d49dc9a9718b7aa8aa4a9 => generated 611 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:15.599164 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31cfd0-1124-4a1b-8739-d005f16b9c18 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:15.712459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31cfd0-1124-4a1b-8739-d005f16b9c18 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:15.758939 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64091287-c1cf-46b4-923b-52540bad99ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:15.759754 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64091287-c1cf-46b4-923b-52540bad99ef None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:15.760041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64091287-c1cf-46b4-923b-52540bad99ef None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:15.760320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64091287-c1cf-46b4-923b-52540bad99ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:15.774603 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64091287-c1cf-46b4-923b-52540bad99ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:15.782793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-64091287-c1cf-46b4-923b-52540bad99ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:27:15.782793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:15.782793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:15.782793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:15.782793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:15.782793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:15.782793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:15.782793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:15.782793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:15.782793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:15.782793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:15.782793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:15.782793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:15.782793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:15.782793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:15.782793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:15.783757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:15.783757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:15.783757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:15.783757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:15.783757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:15.783757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:15.783757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:15.783757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:15.783757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:15.783757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:15.783757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:15.783757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:15.783757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:15.783757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:15.783757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:15.783757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:15.784567 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:15.784567 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:15.784567 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:15.784567 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:15.784567 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:15.784567 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:15.784567 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:15.784567 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:15.784567 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:15.784567 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:15.784567 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:27:15.784567 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:15.784567 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 214/434] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:15.790323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac88ff-8a98-46da-bf2d-a3e1f2c7d484 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:15.897558 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac88ff-8a98-46da-bf2d-a3e1f2c7d484 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:15.912286 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:15.934212 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808ae4a-947e-486c-9e40-5c416da8ab91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:15.934889 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808ae4a-947e-486c-9e40-5c416da8ab91 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:15.935176 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808ae4a-947e-486c-9e40-5c416da8ab91 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:15.935458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808ae4a-947e-486c-9e40-5c416da8ab91 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:15.936220 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808ae4a-947e-486c-9e40-5c416da8ab91 None admin] RBAC: Adding query filter params (name=member) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:15.936530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808ae4a-947e-486c-9e40-5c416da8ab91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:15.938034 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808ae4a-947e-486c-9e40-5c416da8ab91 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:15.948301 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 221/435] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:27:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:15.953830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64091287-c1cf-46b4-923b-52540bad99ef None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:15.983290 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c79028-4654-428c-934e-60f849efe056 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:15.983582 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c79028-4654-428c-934e-60f849efe056 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:15.983862 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c79028-4654-428c-934e-60f849efe056 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:15.985542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 222/436] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:15.990365 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eade9a-58f3-4631-a0c0-2021af5ed29b None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:15.990821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eade9a-58f3-4631-a0c0-2021af5ed29b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:15.991276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eade9a-58f3-4631-a0c0-2021af5ed29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:15.993728 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:16.012280 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-a0eade9a-58f3-4631-a0c0-2021af5ed29b None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:16.013731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a0eade9a-58f3-4631-a0c0-2021af5ed29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:16.084558 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64091287-c1cf-46b4-923b-52540bad99ef None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:16.100623 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 223/437] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:27:16.125229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eade9a-58f3-4631-a0c0-2021af5ed29b None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:16.131064 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186ecb2-7c4f-447b-8898-b5682d994987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:16.131722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186ecb2-7c4f-447b-8898-b5682d994987 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:16.132008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186ecb2-7c4f-447b-8898-b5682d994987 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:16.132289 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186ecb2-7c4f-447b-8898-b5682d994987 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:16.212278 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186ecb2-7c4f-447b-8898-b5682d994987 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:16.213764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186ecb2-7c4f-447b-8898-b5682d994987 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:16.249233 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eade9a-58f3-4631-a0c0-2021af5ed29b None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:16.254740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:16.256686 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:16.290138 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8326622-cc32-4b50-b4d0-cbb81e8133f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:16.291016 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8326622-cc32-4b50-b4d0-cbb81e8133f5 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:16.291327 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8326622-cc32-4b50-b4d0-cbb81e8133f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:16.291567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8326622-cc32-4b50-b4d0-cbb81e8133f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:16.303588 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8326622-cc32-4b50-b4d0-cbb81e8133f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:16.312379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-e8326622-cc32-4b50-b4d0-cbb81e8133f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:27:16.312379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:16.312379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:16.312379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:16.312379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:16.312379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:16.312379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:16.312379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:16.312379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:16.312379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:16.312379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:16.312379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:16.312379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:16.312379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:16.312379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:16.312379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:16.313715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:16.313715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:16.313715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:16.313715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:16.313715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:16.313715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:16.313715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:16.313715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:16.313715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:16.313715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:16.313715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:16.313715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:16.313715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:16.313715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:16.313715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:16.313715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:16.314541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:16.314541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:16.314541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:16.314541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:16.314541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:16.314541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:16.314541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:16.314541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:16.314541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:16.314541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:16.314541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:27:16.314541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:16.315625 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 224/438] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:27:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:27:16.317127 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d56cb4-3f3d-4644-81e0-4f28b372db80 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:16.317127 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d56cb4-3f3d-4644-81e0-4f28b372db80 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:16.317316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d56cb4-3f3d-4644-81e0-4f28b372db80 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:16.319023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 225/439] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:16.324486 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d56cb4-3f3d-4644-81e0-4f28b372db80 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:16.344957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 215/440] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:16.346248 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474181b-3ceb-4f9c-a45e-cf4f2c814679 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:16.346540 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474181b-3ceb-4f9c-a45e-cf4f2c814679 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:16.346817 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474181b-3ceb-4f9c-a45e-cf4f2c814679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:16.362852 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-3474181b-3ceb-4f9c-a45e-cf4f2c814679 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:16.364169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3474181b-3ceb-4f9c-a45e-cf4f2c814679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:16.429423 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d56cb4-3f3d-4644-81e0-4f28b372db80 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:16.432101 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 216/441] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:16.432963 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474181b-3ceb-4f9c-a45e-cf4f2c814679 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:16.465463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e936af-a3e9-4be3-b6e1-167421d4a00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:16.466140 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e936af-a3e9-4be3-b6e1-167421d4a00f None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:16.466432 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e936af-a3e9-4be3-b6e1-167421d4a00f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:16.466717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e936af-a3e9-4be3-b6e1-167421d4a00f None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:16.467477 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e936af-a3e9-4be3-b6e1-167421d4a00f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:16.467790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e936af-a3e9-4be3-b6e1-167421d4a00f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:16.469212 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e936af-a3e9-4be3-b6e1-167421d4a00f None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:16.478885 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 226/442] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:16.479611 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e936af-a3e9-4be3-b6e1-167421d4a00f None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:16.491160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:16.507443 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:16.531251 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474181b-3ceb-4f9c-a45e-cf4f2c814679 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:16.564316 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c90a7-3119-47b2-be95-20d5af240a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:16.565196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c90a7-3119-47b2-be95-20d5af240a39 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:16.565480 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c90a7-3119-47b2-be95-20d5af240a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:16.565753 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c90a7-3119-47b2-be95-20d5af240a39 None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:16.566915 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c90a7-3119-47b2-be95-20d5af240a39 None admin] RBAC: Authorizing `identity:get_project(project_id=debcfc4c17a34d8a80f13016124abfaf)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:16.568348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c90a7-3119-47b2-be95-20d5af240a39 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:16.570035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 217/443] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:27:16 2022] GET /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:16.571306 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c90a7-3119-47b2-be95-20d5af240a39 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:16.578856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e936af-a3e9-4be3-b6e1-167421d4a00f None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:16.611972 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d3a5d-2748-4b44-accc-5112aa6de1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:16.612795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d3a5d-2748-4b44-accc-5112aa6de1ec None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:16.613079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d3a5d-2748-4b44-accc-5112aa6de1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:16.613351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d3a5d-2748-4b44-accc-5112aa6de1ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:16.623671 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d3a5d-2748-4b44-accc-5112aa6de1ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:16.631908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-044d3a5d-2748-4b44-accc-5112aa6de1ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:27:16.631908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:16.631908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:16.631908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:16.631908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:16.631908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:16.631908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:16.631908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:16.631908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:16.631908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:16.631908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:16.631908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:16.631908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:16.631908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:16.631908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:16.631908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:16.632837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:16.632837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:16.632837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:16.632837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:16.632837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:16.632837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:16.632837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:16.632837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:16.632837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:16.632837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:16.632837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:16.632837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:16.632837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:16.632837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:16.632837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:16.632837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:16.634121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:16.634121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:16.634121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:16.634121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:16.634121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:16.634121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:16.634121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:16.634121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:16.634121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:16.634121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:16.634121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:27:16.634121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:16.634121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 227/444] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:27:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:16.634772 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d3a5d-2748-4b44-accc-5112aa6de1ec None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:16.673081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c90a7-3119-47b2-be95-20d5af240a39 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:16.699785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:16.707640 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd91e9-3efc-45ae-b725-8d74550b9356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:16.708283 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd91e9-3efc-45ae-b725-8d74550b9356 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:16.708565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd91e9-3efc-45ae-b725-8d74550b9356 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:16.708869 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd91e9-3efc-45ae-b725-8d74550b9356 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:16.724154 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:16.736292 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d3a5d-2748-4b44-accc-5112aa6de1ec None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:16.769664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46398a6-b5bd-4528-8087-d5afcc1aba91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:16.770442 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46398a6-b5bd-4528-8087-d5afcc1aba91 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:16.770726 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46398a6-b5bd-4528-8087-d5afcc1aba91 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:16.771001 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46398a6-b5bd-4528-8087-d5afcc1aba91 None admin] PATH_INFO: `/v3/groups/fe95c2a6f34746a8b469bf15af149d71` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:16.772164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46398a6-b5bd-4528-8087-d5afcc1aba91 None admin] RBAC: Authorizing `identity:get_group(group_id=fe95c2a6f34746a8b469bf15af149d71)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:16.773665 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46398a6-b5bd-4528-8087-d5afcc1aba91 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:16.775260 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 228/445] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 22:27:16 2022] GET /identity/v3/groups/fe95c2a6f34746a8b469bf15af149d71 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:16.775566 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd91e9-3efc-45ae-b725-8d74550b9356 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:16.776017 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46398a6-b5bd-4528-8087-d5afcc1aba91 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:16.777053 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd91e9-3efc-45ae-b725-8d74550b9356 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:16.874200 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46398a6-b5bd-4528-8087-d5afcc1aba91 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:16.876911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 218/446] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:16.877740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd91e9-3efc-45ae-b725-8d74550b9356 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:16.904368 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:16.910166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a272f-306a-4553-aa4f-664f4a706d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:16.910843 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a272f-306a-4553-aa4f-664f4a706d68 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:16.911196 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a272f-306a-4553-aa4f-664f4a706d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:16.920710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a272f-306a-4553-aa4f-664f4a706d68 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:16.920710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a272f-306a-4553-aa4f-664f4a706d68 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:16.920710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a272f-306a-4553-aa4f-664f4a706d68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:16.920710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a272f-306a-4553-aa4f-664f4a706d68 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:16.922814 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 229/447] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:16.923603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a272f-306a-4553-aa4f-664f4a706d68 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:16.943665 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:16.975400 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd91e9-3efc-45ae-b725-8d74550b9356 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:17.008300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bf0c2-6bfd-4670-8066-bb49e374d523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:17.008981 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bf0c2-6bfd-4670-8066-bb49e374d523 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:17.009267 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bf0c2-6bfd-4670-8066-bb49e374d523 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:17.009546 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bf0c2-6bfd-4670-8066-bb49e374d523 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:17.010373 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bf0c2-6bfd-4670-8066-bb49e374d523 None admin] RBAC: Adding query filter params (group.id=fe95c2a6f34746a8b469bf15af149d71, role.id=1305747c745d49dc9a9718b7aa8aa4a9, scope.project.id=debcfc4c17a34d8a80f13016124abfaf) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:17.010686 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bf0c2-6bfd-4670-8066-bb49e374d523 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:17.012128 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bf0c2-6bfd-4670-8066-bb49e374d523 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:17.017848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 219/448] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 23 22:27:16 2022] GET /identity/v3/role_assignments?group.id=fe95c2a6f34746a8b469bf15af149d71&scope.project.id=debcfc4c17a34d8a80f13016124abfaf&role.id=1305747c745d49dc9a9718b7aa8aa4a9 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:17.018614 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bf0c2-6bfd-4670-8066-bb49e374d523 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:17.034603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a272f-306a-4553-aa4f-664f4a706d68 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:17.075777 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c016a-d1af-49e5-9c57-e5a363cf3347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:17.076687 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c016a-d1af-49e5-9c57-e5a363cf3347 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:17.076992 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c016a-d1af-49e5-9c57-e5a363cf3347 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:17.077254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c016a-d1af-49e5-9c57-e5a363cf3347 None admin] PATH_INFO: `/v3/projects/3f0d9b0cc6e44015a1eedda4627f582b` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:17.078521 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c016a-d1af-49e5-9c57-e5a363cf3347 None admin] RBAC: Authorizing `identity:get_project(project_id=3f0d9b0cc6e44015a1eedda4627f582b)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:17.080000 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c016a-d1af-49e5-9c57-e5a363cf3347 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:17.081996 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 230/449] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:27:16 2022] GET /identity/v3/projects/3f0d9b0cc6e44015a1eedda4627f582b => generated 344 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:17.088255 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c016a-d1af-49e5-9c57-e5a363cf3347 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:17.125577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:17.136271 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bf0c2-6bfd-4670-8066-bb49e374d523 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:17.174958 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddf8d6-b3a0-4a12-963c-05c014eb0519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:17.174958 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddf8d6-b3a0-4a12-963c-05c014eb0519 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:17.176325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddf8d6-b3a0-4a12-963c-05c014eb0519 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:17.177425 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddf8d6-b3a0-4a12-963c-05c014eb0519 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:17.189925 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:17.209459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c016a-d1af-49e5-9c57-e5a363cf3347 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:17.249505 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c33008-e7a9-4bb8-99d2-bc3cee9e2958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:17.250376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c33008-e7a9-4bb8-99d2-bc3cee9e2958 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:17.250903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c33008-e7a9-4bb8-99d2-bc3cee9e2958 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:17.251010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c33008-e7a9-4bb8-99d2-bc3cee9e2958 None admin] PATH_INFO: `/v3/groups/ddc2bb71b27848ebace38b23499197e4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:17.252124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c33008-e7a9-4bb8-99d2-bc3cee9e2958 None admin] RBAC: Authorizing `identity:get_group(group_id=ddc2bb71b27848ebace38b23499197e4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:17.254300 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c33008-e7a9-4bb8-99d2-bc3cee9e2958 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:17.256089 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 231/450] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 22:27:17 2022] GET /identity/v3/groups/ddc2bb71b27848ebace38b23499197e4 => generated 229 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:17.261947 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c33008-e7a9-4bb8-99d2-bc3cee9e2958 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:17.265781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ddf8d6-b3a0-4a12-963c-05c014eb0519 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:17.267291 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ddf8d6-b3a0-4a12-963c-05c014eb0519 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:17.379322 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:17.391980 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 220/451] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:17.396652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c33008-e7a9-4bb8-99d2-bc3cee9e2958 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:17.397541 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddf8d6-b3a0-4a12-963c-05c014eb0519 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:17.430610 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d6371-0107-4e8d-89a2-985e64af139f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:17.431264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d6371-0107-4e8d-89a2-985e64af139f None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:17.431550 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d6371-0107-4e8d-89a2-985e64af139f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:17.431826 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d6371-0107-4e8d-89a2-985e64af139f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:17.432681 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d6371-0107-4e8d-89a2-985e64af139f None admin] RBAC: Adding query filter params (group.id=ddc2bb71b27848ebace38b23499197e4, role.id=6ff7708b59984b9287cc81d46a57b4b4, scope.project.id=3f0d9b0cc6e44015a1eedda4627f582b) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:17.433001 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d6371-0107-4e8d-89a2-985e64af139f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:17.434431 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d6371-0107-4e8d-89a2-985e64af139f None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:17.440128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 232/452] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 23 22:27:17 2022] GET /identity/v3/role_assignments?group.id=ddc2bb71b27848ebace38b23499197e4&scope.project.id=3f0d9b0cc6e44015a1eedda4627f582b&role.id=6ff7708b59984b9287cc81d46a57b4b4 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:27:17.500863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddf8d6-b3a0-4a12-963c-05c014eb0519 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:17.536099 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b5070-dd2b-4c91-966d-73dbeee90da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:17.537012 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b5070-dd2b-4c91-966d-73dbeee90da0 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:17.537300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b5070-dd2b-4c91-966d-73dbeee90da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:17.537578 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b5070-dd2b-4c91-966d-73dbeee90da0 None admin] PATH_INFO: `/v3/projects/debcfc4c17a34d8a80f13016124abfaf` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:17.538848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b5070-dd2b-4c91-966d-73dbeee90da0 None admin] RBAC: Authorizing `identity:get_project(project_id=debcfc4c17a34d8a80f13016124abfaf)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:17.540947 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b5070-dd2b-4c91-966d-73dbeee90da0 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:17.542939 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 221/453] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:27:17 2022] GET /identity/v3/projects/debcfc4c17a34d8a80f13016124abfaf => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:27:17.548604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d6371-0107-4e8d-89a2-985e64af139f None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:17.569417 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:17.652326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d6371-0107-4e8d-89a2-985e64af139f None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:17.684593 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17926db-bd12-4bd9-83be-3481f170c9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:17.685444 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17926db-bd12-4bd9-83be-3481f170c9f7 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:17.685729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17926db-bd12-4bd9-83be-3481f170c9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:17.686008 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17926db-bd12-4bd9-83be-3481f170c9f7 None admin] PATH_INFO: `/v3/groups/fe95c2a6f34746a8b469bf15af149d71` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:17.687155 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17926db-bd12-4bd9-83be-3481f170c9f7 None admin] RBAC: Authorizing `identity:get_group(group_id=fe95c2a6f34746a8b469bf15af149d71)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:17.688613 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17926db-bd12-4bd9-83be-3481f170c9f7 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:17.690201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 233/454] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 23 22:27:17 2022] GET /identity/v3/groups/fe95c2a6f34746a8b469bf15af149d71 => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:17.696710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b5070-dd2b-4c91-966d-73dbeee90da0 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:17.752852 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:17.793117 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b5070-dd2b-4c91-966d-73dbeee90da0 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:17.825602 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8264aa-291a-4c0b-b12e-e0ea722de1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:17.826239 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8264aa-291a-4c0b-b12e-e0ea722de1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:17.826525 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8264aa-291a-4c0b-b12e-e0ea722de1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:17.826804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8264aa-291a-4c0b-b12e-e0ea722de1b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:17.827633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8264aa-291a-4c0b-b12e-e0ea722de1b2 None admin] RBAC: Adding query filter params (group.id=fe95c2a6f34746a8b469bf15af149d71, role.id=c3ad3cf6be504e35aaec10b7b9f3f34d, scope.project.id=debcfc4c17a34d8a80f13016124abfaf) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:17.827951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8264aa-291a-4c0b-b12e-e0ea722de1b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:17.829407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8264aa-291a-4c0b-b12e-e0ea722de1b2 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:17.835022 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 222/455] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 23 22:27:17 2022] GET /identity/v3/role_assignments?group.id=fe95c2a6f34746a8b469bf15af149d71&scope.project.id=debcfc4c17a34d8a80f13016124abfaf&role.id=c3ad3cf6be504e35aaec10b7b9f3f34d => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:20.467004 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788281d8-61ac-43c9-8ca5-aa56ba83eff7 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:20.467865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788281d8-61ac-43c9-8ca5-aa56ba83eff7 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:20.468317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788281d8-61ac-43c9-8ca5-aa56ba83eff7 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:20.469837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 234/456] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:20.474928 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e2b37-1274-4363-a594-61482a193e66 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:20.475663 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e2b37-1274-4363-a594-61482a193e66 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:20.476108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e2b37-1274-4363-a594-61482a193e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:20.501587 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-bc5e2b37-1274-4363-a594-61482a193e66 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:20.503014 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-bc5e2b37-1274-4363-a594-61482a193e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:20.591278 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 223/457] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:20.641145 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788281d8-61ac-43c9-8ca5-aa56ba83eff7 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:20.668016 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee92bad-4491-4bc2-91ba-5e01d77da6df None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:20.668333 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee92bad-4491-4bc2-91ba-5e01d77da6df None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:20.668604 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee92bad-4491-4bc2-91ba-5e01d77da6df None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:20.670103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 224/458] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:27:20.688760 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c1133-6b7e-4412-9487-0c7f75e551f2 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:20.689378 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c1133-6b7e-4412-9487-0c7f75e551f2 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:20.689857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c1133-6b7e-4412-9487-0c7f75e551f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:20.718414 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-491c1133-6b7e-4412-9487-0c7f75e551f2 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:20.720764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-491c1133-6b7e-4412-9487-0c7f75e551f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:20.778741 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:20.792987 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:20.796274 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788281d8-61ac-43c9-8ca5-aa56ba83eff7 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:20.811619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 225/459] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:27:20.814232 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d774fd2-b1ea-495c-a6d4-c51e69544d42 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:20.814670 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d774fd2-b1ea-495c-a6d4-c51e69544d42 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:20.815188 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d774fd2-b1ea-495c-a6d4-c51e69544d42 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:20.816938 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 226/460] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:20.823383 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b823ad2-452b-4396-8793-35f27f4d0298 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:20.823856 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b823ad2-452b-4396-8793-35f27f4d0298 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:20.824275 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b823ad2-452b-4396-8793-35f27f4d0298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:20.834278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7d160-23c9-4b33-8093-0a21cdf4c7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:20.835213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7d160-23c9-4b33-8093-0a21cdf4c7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:20.835600 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7d160-23c9-4b33-8093-0a21cdf4c7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:20.836005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7d160-23c9-4b33-8093-0a21cdf4c7c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:20.843063 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-3b823ad2-452b-4396-8793-35f27f4d0298 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:20.844521 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3b823ad2-452b-4396-8793-35f27f4d0298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:20.849938 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7d160-23c9-4b33-8093-0a21cdf4c7c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:20.851621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7d160-23c9-4b33-8093-0a21cdf4c7c6 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:20.875905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-e5f7d160-23c9-4b33-8093-0a21cdf4c7c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:20.875905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:20.875905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:20.875905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:20.875905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:20.875905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:20.875905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:20.875905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:20.875905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:20.875905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:20.875905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:20.875905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:20.875905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:20.875905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:20.875905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:20.877014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:20.877014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:20.877014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:20.877014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:20.877014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:20.877014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:20.877014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:20.877014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:20.877014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:20.877014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:20.877014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:20.877014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:20.877014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:20.877014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:20.877014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:20.877014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:20.877014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:20.877946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:20.877946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:20.877946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:20.877946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:20.884808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 235/461] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:20.886736 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7d160-23c9-4b33-8093-0a21cdf4c7c6 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:20.932167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 227/462] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:27:20.933281 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b823ad2-452b-4396-8793-35f27f4d0298 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:21.024921 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7d160-23c9-4b33-8093-0a21cdf4c7c6 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:21.062035 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72196437-5756-42b6-b6f3-19da846d3e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:21.063184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72196437-5756-42b6-b6f3-19da846d3e03 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:21.063685 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72196437-5756-42b6-b6f3-19da846d3e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:21.064155 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72196437-5756-42b6-b6f3-19da846d3e03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:21.065808 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b823ad2-452b-4396-8793-35f27f4d0298 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:21.072059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:21.076860 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72196437-5756-42b6-b6f3-19da846d3e03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:21.078509 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72196437-5756-42b6-b6f3-19da846d3e03 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:21.083757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:21.096765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-72196437-5756-42b6-b6f3-19da846d3e03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:21.096765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:21.096765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:21.096765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:21.096765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:21.096765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:21.096765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:21.096765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:21.096765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:21.096765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:21.096765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:21.096765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:21.096765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:21.096765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:21.096765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:21.097622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:21.097622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:21.097622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:21.097622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:21.097622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:21.097622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:21.097622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:21.097622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:21.097622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:21.097622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:21.097622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:21.097622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:21.097622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:21.097622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:21.097622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:21.097622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:21.097622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:21.098455 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:21.098455 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:21.098455 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:21.098455 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:21.100757 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 236/463] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:21.102204 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72196437-5756-42b6-b6f3-19da846d3e03 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:21.106257 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d627cc-45aa-48de-af8c-eac598bca4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:21.106931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d627cc-45aa-48de-af8c-eac598bca4ef None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:21.107209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d627cc-45aa-48de-af8c-eac598bca4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:21.107493 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d627cc-45aa-48de-af8c-eac598bca4ef None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:21.108221 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d627cc-45aa-48de-af8c-eac598bca4ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:21.108531 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d627cc-45aa-48de-af8c-eac598bca4ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:21.110208 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d627cc-45aa-48de-af8c-eac598bca4ef None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:21.129752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 228/464] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:21.131000 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb2933-6e36-47f6-b8c2-f0a168df3fb9 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:21.131301 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb2933-6e36-47f6-b8c2-f0a168df3fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:21.131583 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb2933-6e36-47f6-b8c2-f0a168df3fb9 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:21.137434 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 229/465] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:21.138252 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb2933-6e36-47f6-b8c2-f0a168df3fb9 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:21.216108 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72196437-5756-42b6-b6f3-19da846d3e03 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:21.238351 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb2933-6e36-47f6-b8c2-f0a168df3fb9 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:21.251263 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b56ed-9908-4262-8788-aa13dc901b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:21.252348 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b56ed-9908-4262-8788-aa13dc901b7f None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:21.252852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b56ed-9908-4262-8788-aa13dc901b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:21.253283 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b56ed-9908-4262-8788-aa13dc901b7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:21.265458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b56ed-9908-4262-8788-aa13dc901b7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:21.272509 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b56ed-9908-4262-8788-aa13dc901b7f None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:21.273892 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80ab60-fe36-4a23-bc2a-33b0038e845a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:21.274572 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80ab60-fe36-4a23-bc2a-33b0038e845a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:21.274974 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80ab60-fe36-4a23-bc2a-33b0038e845a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:21.275253 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80ab60-fe36-4a23-bc2a-33b0038e845a None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:21.275980 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80ab60-fe36-4a23-bc2a-33b0038e845a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:21.276286 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80ab60-fe36-4a23-bc2a-33b0038e845a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:21.278290 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80ab60-fe36-4a23-bc2a-33b0038e845a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:21.295818 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 230/466] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:21.295818 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80ab60-fe36-4a23-bc2a-33b0038e845a None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:21.296027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-144b56ed-9908-4262-8788-aa13dc901b7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:21.296027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:21.296027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:21.296027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:21.296027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:21.296027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:21.296027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:21.296027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:21.296027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:21.296027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:21.296027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:21.296027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:21.296027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:21.296027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:21.296027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:21.297086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:21.297086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:21.297086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:21.297086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:21.297086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:21.297086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:21.297086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:21.297086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:21.297086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:21.297086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:21.297086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:21.297086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:21.297086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:21.297086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:21.297086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:21.297086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:21.297086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:21.297955 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:21.297955 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:21.297955 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:21.297955 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:21.300912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 237/467] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:21.302168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8876d3-9deb-42e4-8c7b-e526909cfecc None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:21.302458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8876d3-9deb-42e4-8c7b-e526909cfecc None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:21.302753 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8876d3-9deb-42e4-8c7b-e526909cfecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:21.319265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-bf8876d3-9deb-42e4-8c7b-e526909cfecc None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:21.320979 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-bf8876d3-9deb-42e4-8c7b-e526909cfecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:21.331425 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:21.358116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:21.393662 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 238/468] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:21.394814 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8876d3-9deb-42e4-8c7b-e526909cfecc None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:21.413473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80ab60-fe36-4a23-bc2a-33b0038e845a None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:21.454267 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3592ccb-da2e-4cdc-957a-18ef868ab7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:21.455229 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3592ccb-da2e-4cdc-957a-18ef868ab7d8 None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:21.455713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3592ccb-da2e-4cdc-957a-18ef868ab7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:21.455999 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3592ccb-da2e-4cdc-957a-18ef868ab7d8 None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:21.456747 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3592ccb-da2e-4cdc-957a-18ef868ab7d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:21.459354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3592ccb-da2e-4cdc-957a-18ef868ab7d8 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:21.499987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8876d3-9deb-42e4-8c7b-e526909cfecc None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:21.533338 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 231/469] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 22:27:21 2022] POST /identity/v3/users => generated 246 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:27:21.534358 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3592ccb-da2e-4cdc-957a-18ef868ab7d8 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:21.545727 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8cee03-9e52-4c51-9107-3f32d215ad81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:21.546677 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cee03-9e52-4c51-9107-3f32d215ad81 None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:21.547146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cee03-9e52-4c51-9107-3f32d215ad81 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:21.547566 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cee03-9e52-4c51-9107-3f32d215ad81 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:21.554406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8cee03-9e52-4c51-9107-3f32d215ad81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:21.554406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8cee03-9e52-4c51-9107-3f32d215ad81 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:21.641234 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 239/470] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 22:27:21 2022] POST /identity/v3/users => generated 248 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:27:21.642545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8cee03-9e52-4c51-9107-3f32d215ad81 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:21.657299 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:21.670969 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3592ccb-da2e-4cdc-957a-18ef868ab7d8 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:21.706970 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:21.725198 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c38a2f3-a476-481c-8cb5-9eff19d7c73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:21.725893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38a2f3-a476-481c-8cb5-9eff19d7c73a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:21.726491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38a2f3-a476-481c-8cb5-9eff19d7c73a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:21.727208 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38a2f3-a476-481c-8cb5-9eff19d7c73a None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:21.727965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c38a2f3-a476-481c-8cb5-9eff19d7c73a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:21.728279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c38a2f3-a476-481c-8cb5-9eff19d7c73a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:21.730392 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c38a2f3-a476-481c-8cb5-9eff19d7c73a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:21.752664 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 232/471] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:21.755080 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c38a2f3-a476-481c-8cb5-9eff19d7c73a None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:21.815987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8cee03-9e52-4c51-9107-3f32d215ad81 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:21.865907 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5659c-3785-44eb-a5a4-30248a8f4d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:21.867094 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5659c-3785-44eb-a5a4-30248a8f4d27 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:21.867934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5659c-3785-44eb-a5a4-30248a8f4d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:21.868396 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5659c-3785-44eb-a5a4-30248a8f4d27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:21.879700 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5659c-3785-44eb-a5a4-30248a8f4d27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:21.881341 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5659c-3785-44eb-a5a4-30248a8f4d27 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:21.888875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c38a2f3-a476-481c-8cb5-9eff19d7c73a None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:21.902604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-ebc5659c-3785-44eb-a5a4-30248a8f4d27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:21.902604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:21.902604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:21.902604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:21.902604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:21.902604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:21.902604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:21.902604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:21.902604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:21.902604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:21.902604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:21.902604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:21.902604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:21.902604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:21.902604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:21.903490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:21.903490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:21.903490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:21.903490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:21.903490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:21.903490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:21.903490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:21.903490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:21.903490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:21.903490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:21.903490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:21.903490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:21.903490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:21.903490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:21.903490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:21.903490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:21.903490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:21.904329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:21.904329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:21.904329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:21.904329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:21.905144 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 240/472] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:21.909762 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5659c-3785-44eb-a5a4-30248a8f4d27 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:21.940419 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95887468-526d-4e8a-a0d0-ef4d27c83c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:21.941129 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95887468-526d-4e8a-a0d0-ef4d27c83c88 None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:21.941419 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95887468-526d-4e8a-a0d0-ef4d27c83c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:21.941699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95887468-526d-4e8a-a0d0-ef4d27c83c88 None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:21.942428 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95887468-526d-4e8a-a0d0-ef4d27c83c88 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:21.943880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95887468-526d-4e8a-a0d0-ef4d27c83c88 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:22.018219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:22.036782 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5659c-3785-44eb-a5a4-30248a8f4d27 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:22.053941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 233/473] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 22:27:21 2022] POST /identity/v3/users => generated 248 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:27:22.056035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cff7e-b04c-408f-a66d-182d765c9391 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:22.056339 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cff7e-b04c-408f-a66d-182d765c9391 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:22.056654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cff7e-b04c-408f-a66d-182d765c9391 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:22.058247 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 234/474] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:22.064106 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd02cb1-cc60-4b43-a8b0-6e5d9ed9b9cd None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:22.064417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd02cb1-cc60-4b43-a8b0-6e5d9ed9b9cd None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:22.064706 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd02cb1-cc60-4b43-a8b0-6e5d9ed9b9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:22.078256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:22.079445 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90283ecd-0c94-4729-b1c9-91ff6d054f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:22.080122 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90283ecd-0c94-4729-b1c9-91ff6d054f1e None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:22.080545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90283ecd-0c94-4729-b1c9-91ff6d054f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:22.080996 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90283ecd-0c94-4729-b1c9-91ff6d054f1e None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:22.081749 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90283ecd-0c94-4729-b1c9-91ff6d054f1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:22.082411 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90283ecd-0c94-4729-b1c9-91ff6d054f1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:22.083878 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90283ecd-0c94-4729-b1c9-91ff6d054f1e None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:22.085035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-5bd02cb1-cc60-4b43-a8b0-6e5d9ed9b9cd None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:22.087515 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-5bd02cb1-cc60-4b43-a8b0-6e5d9ed9b9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:22.100630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 241/475] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:27:22.111350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90283ecd-0c94-4729-b1c9-91ff6d054f1e None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:22.228042 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 235/476] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:22.257732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd02cb1-cc60-4b43-a8b0-6e5d9ed9b9cd None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:22.266152 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90283ecd-0c94-4729-b1c9-91ff6d054f1e None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:22.306989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc47a3a-f2a1-4340-ab3f-72e93296d0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:22.307661 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc47a3a-f2a1-4340-ab3f-72e93296d0f9 None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:22.307942 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc47a3a-f2a1-4340-ab3f-72e93296d0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:22.308217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc47a3a-f2a1-4340-ab3f-72e93296d0f9 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:22.309049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc47a3a-f2a1-4340-ab3f-72e93296d0f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:22.310387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc47a3a-f2a1-4340-ab3f-72e93296d0f9 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:22.390359 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:22.404820 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd02cb1-cc60-4b43-a8b0-6e5d9ed9b9cd None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:22.448039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8ca9b-7b64-4f4d-827b-3e4f106ee5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:22.448749 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8ca9b-7b64-4f4d-827b-3e4f106ee5bf None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:22.449042 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8ca9b-7b64-4f4d-827b-3e4f106ee5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:22.449325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8ca9b-7b64-4f4d-827b-3e4f106ee5bf None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:22.450079 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd8ca9b-7b64-4f4d-827b-3e4f106ee5bf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:22.451455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd8ca9b-7b64-4f4d-827b-3e4f106ee5bf None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:22.454871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:22.485125 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 242/477] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 22:27:22 2022] POST /identity/v3/users => generated 249 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:27:22.521774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.sql.core [None req-4cd8ca9b-7b64-4f4d-827b-3e4f106ee5bf None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 22:27:22.521774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 22:27:22.521774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [parameters: {'id': '007aa17493cd42fc9b03b808f844840f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 22:27:22.521774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80519) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 22:27:22.544448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-4cd8ca9b-7b64-4f4d-827b-3e4f106ee5bf None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 22:27:22.544448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:22.544448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 22:27:22.544448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 22:27:22.544448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 22:27:22.544448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 22:27:22.544448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 22:27:22.544448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 22:27:22.544448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 22:27:22.544448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application conn.query(q) Jun 23 22:27:22.544448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 22:27:22.544448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 22:27:22.544448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 22:27:22.544448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application result.read() Jun 23 22:27:22.544448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 22:27:22.545810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 22:27:22.545810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 22:27:22.545810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 22:27:22.545810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 22:27:22.545810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 22:27:22.545810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 22:27:22.545810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 22:27:22.545810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 22:27:22.545810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:22.545810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 22:27:22.545810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:22.545810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:22.545810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 22:27:22.545810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 22:27:22.545810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 23 22:27:22.545810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return ref.to_dict() Jun 23 22:27:22.545810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 22:27:22.545810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application next(self.gen) Jun 23 22:27:22.546721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 22:27:22.546721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application yield resource Jun 23 22:27:22.546721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 22:27:22.546721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application next(self.gen) Jun 23 22:27:22.546721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 22:27:22.546721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 22:27:22.546721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 22:27:22.546721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 22:27:22.546721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 22:27:22.546721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise self.value Jun 23 22:27:22.546721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 22:27:22.546721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 22:27:22.546721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 22:27:22.546721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application session.commit() Jun 23 22:27:22.546721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 22:27:22.546721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 22:27:22.546721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 22:27:22.547585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 22:27:22.547585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 22:27:22.547585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self.session.flush() Jun 23 22:27:22.547585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 22:27:22.547585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 22:27:22.547585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 22:27:22.547585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 22:27:22.547585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 22:27:22.547585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application compat.raise_( Jun 23 22:27:22.547585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 22:27:22.547585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception Jun 23 22:27:22.547585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 22:27:22.547585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 22:27:22.547585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 22:27:22.547585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 22:27:22.547585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 22:27:22.547585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 22:27:22.548468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 22:27:22.548468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 22:27:22.548468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 22:27:22.548468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 22:27:22.548468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 22:27:22.548468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 22:27:22.548468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 22:27:22.548468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 22:27:22.548468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 22:27:22.548468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 22:27:22.548468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 22:27:22.548468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 22:27:22.548468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 22:27:22.548468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 22:27:22.548468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 22:27:22.548468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception Jun 23 22:27:22.551496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 22:27:22.551496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 22:27:22.551496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 22:27:22.551496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 22:27:22.551496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 22:27:22.551496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 22:27:22.551496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 22:27:22.551496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application conn.query(q) Jun 23 22:27:22.551496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 22:27:22.551496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 22:27:22.551496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 22:27:22.551496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application result.read() Jun 23 22:27:22.551496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 22:27:22.551496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 22:27:22.551496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 22:27:22.551496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 22:27:22.553508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 22:27:22.553508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 22:27:22.553508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 22:27:22.553508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 22:27:22.553508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 22:27:22.553508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 22:27:22.553508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [parameters: {'id': '007aa17493cd42fc9b03b808f844840f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 22:27:22.553508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 22:27:22.553508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:22.553508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:27:22.553508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:22.553508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:22.553508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:22.553508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:22.553508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:22.553508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:22.556721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:22.556721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:22.556721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:22.556721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:22.556721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:22.556721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:22.556721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 23 22:27:22.556721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 23 22:27:22.556721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:22.556721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:22.556721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 23 22:27:22.556721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 23 22:27:22.556721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 22:27:22.556721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 22:27:22.556721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 22:27:22.556721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:22.559095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 236/478] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Jun 23 22:27:22 2022] POST /identity/v3/roles => generated 144 bytes in 295 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:27:22.560717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc47a3a-f2a1-4340-ab3f-72e93296d0f9 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:22.719739 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc47a3a-f2a1-4340-ab3f-72e93296d0f9 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:22.774438 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f892a5a-2aca-4dfb-8895-180480dd8c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:22.775480 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f892a5a-2aca-4dfb-8895-180480dd8c0a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:22.775929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f892a5a-2aca-4dfb-8895-180480dd8c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:22.776348 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f892a5a-2aca-4dfb-8895-180480dd8c0a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:22.788568 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f892a5a-2aca-4dfb-8895-180480dd8c0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:22.797742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-9f892a5a-2aca-4dfb-8895-180480dd8c0a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:27:22.797742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:22.797742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:22.797742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:22.797742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:22.797742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:22.797742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:22.797742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:22.797742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:22.797742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:22.797742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:22.797742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:22.797742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:22.797742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:22.797742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:22.797742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:22.798827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:22.798827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:22.798827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:22.798827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:22.798827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:22.798827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:22.798827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:22.798827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:22.798827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:22.798827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:22.798827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:22.798827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:22.798827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:22.798827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:22.798827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:22.798827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:22.799788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:22.799788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:22.799788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:22.799788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:22.799788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:22.799788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:22.799788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:22.799788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:22.799788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:22.799788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:22.799788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:27:22.799788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:22.802151 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 243/479] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:27:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:22.808025 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8ca9b-7b64-4f4d-827b-3e4f106ee5bf None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:22.926931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8ca9b-7b64-4f4d-827b-3e4f106ee5bf None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:22.971231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdbf905-4bf6-4997-920a-946660758b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:22.971921 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbf905-4bf6-4997-920a-946660758b5a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:22.972209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbf905-4bf6-4997-920a-946660758b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:22.972487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbf905-4bf6-4997-920a-946660758b5a None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:22.973335 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdbf905-4bf6-4997-920a-946660758b5a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:22.973645 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdbf905-4bf6-4997-920a-946660758b5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:22.975066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdbf905-4bf6-4997-920a-946660758b5a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:22.988509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 237/480] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:24.000003 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652488f-f8af-4387-b261-c47a26f44b29 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:24.000541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652488f-f8af-4387-b261-c47a26f44b29 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:24.000541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652488f-f8af-4387-b261-c47a26f44b29 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:24.002189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 244/481] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:24.009046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402b8f8-2bdd-44c0-951c-8b1171defabc None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:24.009461 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402b8f8-2bdd-44c0-951c-8b1171defabc None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:24.009633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402b8f8-2bdd-44c0-951c-8b1171defabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:24.033695 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-f402b8f8-2bdd-44c0-951c-8b1171defabc None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:24.034886 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-f402b8f8-2bdd-44c0-951c-8b1171defabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:24.110818 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:24.123786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 238/482] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:27:24.170758 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652488f-f8af-4387-b261-c47a26f44b29 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:24.176359 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ae88d-6293-4ee8-81dd-9a02d83e9a1e None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:24.176675 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ae88d-6293-4ee8-81dd-9a02d83e9a1e None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:24.176962 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ae88d-6293-4ee8-81dd-9a02d83e9a1e None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:24.178618 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 239/483] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:27:24.185964 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c486c2-f43d-420e-9594-53cc9eb2d737 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:24.186372 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c486c2-f43d-420e-9594-53cc9eb2d737 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:24.186752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c486c2-f43d-420e-9594-53cc9eb2d737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:24.204423 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-c1c486c2-f43d-420e-9594-53cc9eb2d737 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:24.205766 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c1c486c2-f43d-420e-9594-53cc9eb2d737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:24.270677 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:24.325055 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 240/484] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:27:24.342324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652488f-f8af-4387-b261-c47a26f44b29 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:24.345165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c486c2-f43d-420e-9594-53cc9eb2d737 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:24.413253 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a560ca1-8caa-4949-b366-568c874efa4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:24.414759 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a560ca1-8caa-4949-b366-568c874efa4c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:24.415047 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a560ca1-8caa-4949-b366-568c874efa4c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:24.415436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a560ca1-8caa-4949-b366-568c874efa4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:24.437224 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a560ca1-8caa-4949-b366-568c874efa4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:24.453589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-0a560ca1-8caa-4949-b366-568c874efa4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:24.453589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:24.453589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:24.453589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:24.453589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:24.453589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:24.453589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:24.453589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:24.453589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:24.453589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:24.453589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:24.453589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:24.453589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:24.453589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:24.453589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:24.453589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:24.454515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:24.454515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:24.454515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:24.454515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:24.454515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:24.454515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:24.454515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:24.454515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:24.454515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:24.454515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:24.454515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:24.454515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:24.454515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:24.454515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:24.454515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:24.454515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:24.455316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:24.455316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:24.455316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:24.455316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:24.455316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:24.455316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:24.455316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:24.455316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:24.455316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:24.455316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:24.455316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:24.455316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:24.458056 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 245/485] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:27:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:27:24.464797 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a560ca1-8caa-4949-b366-568c874efa4c None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:24.513005 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c486c2-f43d-420e-9594-53cc9eb2d737 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:24.557135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:24.578810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ed4f7-304f-4ea8-9f43-546c79b3a19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:24.580085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ed4f7-304f-4ea8-9f43-546c79b3a19e None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:24.581750 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ed4f7-304f-4ea8-9f43-546c79b3a19e None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:24.582848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ed4f7-304f-4ea8-9f43-546c79b3a19e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:24.596930 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ed4f7-304f-4ea8-9f43-546c79b3a19e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:24.606348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-330ed4f7-304f-4ea8-9f43-546c79b3a19e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:24.606348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:24.606348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:24.606348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:24.606348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:24.606348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:24.606348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:24.606348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:24.606348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:24.606348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:24.606348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:24.606348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:24.606348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:24.606348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:24.606348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:24.606348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:24.607249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:24.607249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:24.607249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:24.607249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:24.607249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:24.607249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:24.607249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:24.607249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:24.607249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:24.607249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:24.607249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:24.607249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:24.607249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:24.607249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:24.607249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:24.607249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:24.608066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:24.608066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:24.608066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:24.608066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:24.608066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:24.608066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:24.608066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:24.608066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:24.608066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:24.608066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:24.608066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:24.608066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:24.610863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 241/486] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:27:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:24.617830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ed4f7-304f-4ea8-9f43-546c79b3a19e None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:24.656661 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a560ca1-8caa-4949-b366-568c874efa4c None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:24.674305 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:24.692857 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422c7bc-4e9c-423c-98ec-01a395f995dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:24.693534 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422c7bc-4e9c-423c-98ec-01a395f995dc None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:24.693822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422c7bc-4e9c-423c-98ec-01a395f995dc None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:24.694112 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422c7bc-4e9c-423c-98ec-01a395f995dc None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:24.694872 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422c7bc-4e9c-423c-98ec-01a395f995dc None admin] RBAC: Adding query filter params (name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:24.695937 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422c7bc-4e9c-423c-98ec-01a395f995dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:24.697748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422c7bc-4e9c-423c-98ec-01a395f995dc None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:24.706715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 246/487] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:27:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:24.708135 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f75e0f-6bd0-4a37-97ca-2dccba3f3af9 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:24.708484 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f75e0f-6bd0-4a37-97ca-2dccba3f3af9 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:24.708924 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f75e0f-6bd0-4a37-97ca-2dccba3f3af9 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:24.710669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 247/488] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:24.716013 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f75e0f-6bd0-4a37-97ca-2dccba3f3af9 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:24.761935 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ed4f7-304f-4ea8-9f43-546c79b3a19e None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:24.811802 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7462ca-e637-43a8-921f-842b4a9a8369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:24.812982 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7462ca-e637-43a8-921f-842b4a9a8369 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:24.813419 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7462ca-e637-43a8-921f-842b4a9a8369 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:24.814082 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7462ca-e637-43a8-921f-842b4a9a8369 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:24.815412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7462ca-e637-43a8-921f-842b4a9a8369 None admin] RBAC: Adding query filter params (name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:24.815875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7462ca-e637-43a8-921f-842b4a9a8369 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:24.817509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7462ca-e637-43a8-921f-842b4a9a8369 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:24.830411 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 242/489] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:27:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:24.831993 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debdd1d-43d6-4adc-b009-6013a07e7899 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:24.832125 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f75e0f-6bd0-4a37-97ca-2dccba3f3af9 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:24.832605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debdd1d-43d6-4adc-b009-6013a07e7899 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:24.833067 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debdd1d-43d6-4adc-b009-6013a07e7899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:24.855111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-8debdd1d-43d6-4adc-b009-6013a07e7899 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:24.857200 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-8debdd1d-43d6-4adc-b009-6013a07e7899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:24.872011 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cf179-c3a5-4547-91cb-93405f26964f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:24.872962 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cf179-c3a5-4547-91cb-93405f26964f None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:24.873323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cf179-c3a5-4547-91cb-93405f26964f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:24.873677 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cf179-c3a5-4547-91cb-93405f26964f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:24.885379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cf179-c3a5-4547-91cb-93405f26964f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:24.886992 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cf179-c3a5-4547-91cb-93405f26964f None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:24.907426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-081cf179-c3a5-4547-91cb-93405f26964f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:24.907426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:24.907426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:24.907426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:24.907426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:24.907426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:24.907426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:24.907426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:24.907426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:24.907426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:24.907426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:24.907426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:24.907426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:24.907426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:24.907426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:24.908310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:24.908310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:24.908310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:24.908310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:24.908310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:24.908310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:24.908310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:24.908310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:24.908310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:24.908310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:24.908310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:24.908310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:24.908310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:24.908310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:24.908310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:24.908310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:24.908310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:24.909201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:24.909201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:24.909201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:24.909201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:24.909201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 248/490] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:24.910347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cf179-c3a5-4547-91cb-93405f26964f None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:24.916054 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:24.946693 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:24.953668 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 243/491] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:24.956703 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8debdd1d-43d6-4adc-b009-6013a07e7899 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:25.057470 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cf179-c3a5-4547-91cb-93405f26964f None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:25.080312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8debdd1d-43d6-4adc-b009-6013a07e7899 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:25.109090 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f31123-c9a9-4097-959a-93d4fdc52aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:25.110325 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f31123-c9a9-4097-959a-93d4fdc52aec None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:25.110619 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f31123-c9a9-4097-959a-93d4fdc52aec None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:25.110903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f31123-c9a9-4097-959a-93d4fdc52aec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:25.122716 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829e539-f152-405e-8890-991988430d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:25.123693 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829e539-f152-405e-8890-991988430d1b None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:25.124184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829e539-f152-405e-8890-991988430d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:25.124691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829e539-f152-405e-8890-991988430d1b None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:25.124951 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f31123-c9a9-4097-959a-93d4fdc52aec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:25.125912 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829e539-f152-405e-8890-991988430d1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:25.126424 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829e539-f152-405e-8890-991988430d1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:25.126526 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f31123-c9a9-4097-959a-93d4fdc52aec None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:25.128304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829e539-f152-405e-8890-991988430d1b None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:25.146096 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 244/492] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:25.147099 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829e539-f152-405e-8890-991988430d1b None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:25.149629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-f2f31123-c9a9-4097-959a-93d4fdc52aec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:25.149629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:25.149629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:25.149629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:25.149629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:25.149629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:25.149629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:25.149629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:25.149629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:25.149629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:25.149629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:25.149629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:25.149629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:25.149629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:25.149629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:25.150452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:25.150452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:25.150452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:25.150452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:25.150452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:25.150452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:25.150452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:25.150452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:25.150452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:25.150452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:25.150452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:25.150452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:25.150452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:25.150452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:25.150452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:25.150452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:25.150452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:25.151327 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:25.151327 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:25.151327 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:25.151327 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:25.153383 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 249/493] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:25.154331 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f31123-c9a9-4097-959a-93d4fdc52aec None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:25.196355 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:25.252517 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:25.276484 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f31123-c9a9-4097-959a-93d4fdc52aec None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:25.283604 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829e539-f152-405e-8890-991988430d1b None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:25.323845 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73347cb-3bbe-4cc2-a8e6-7da5c4458d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:25.324681 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73347cb-3bbe-4cc2-a8e6-7da5c4458d0a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:25.325194 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73347cb-3bbe-4cc2-a8e6-7da5c4458d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:25.325300 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73347cb-3bbe-4cc2-a8e6-7da5c4458d0a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:25.333214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73347cb-3bbe-4cc2-a8e6-7da5c4458d0a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:25.334930 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73347cb-3bbe-4cc2-a8e6-7da5c4458d0a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:25.339195 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20887a4-efd6-4f1c-9b65-f5fb8106cae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:25.340047 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20887a4-efd6-4f1c-9b65-f5fb8106cae6 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:25.340338 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20887a4-efd6-4f1c-9b65-f5fb8106cae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:25.340686 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20887a4-efd6-4f1c-9b65-f5fb8106cae6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:25.351900 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20887a4-efd6-4f1c-9b65-f5fb8106cae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:25.352531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-b73347cb-3bbe-4cc2-a8e6-7da5c4458d0a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:27:25.352531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:25.352531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:25.352531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:25.352531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:25.352531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:25.352531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:25.352531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:25.352531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:25.352531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:25.352531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:25.352531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:25.352531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:25.352531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:25.352531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:25.352531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:25.353439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:25.353439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:25.353439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:25.353439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:25.353439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:25.353439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:25.353439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:25.353439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:25.353439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:25.353439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:25.353439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:25.353439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:25.353439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:25.353439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:25.353439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:25.353439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:25.353439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:25.353439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:25.354347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:25.354347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:25.354347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:25.354347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:25.354347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:25.354347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:25.354347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:25.354347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:25.354347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:27:25.354347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:25.354347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 250/494] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 23 22:27:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:25.354908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73347cb-3bbe-4cc2-a8e6-7da5c4458d0a None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:25.359904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-d20887a4-efd6-4f1c-9b65-f5fb8106cae6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:25.359904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:25.359904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:25.359904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:25.359904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:25.359904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:25.359904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:25.359904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:25.359904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:25.359904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:25.359904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:25.359904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:25.359904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:25.359904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:25.359904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:25.359904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:25.364116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:25.364116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:25.364116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:25.364116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:25.364116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:25.364116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:25.364116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:25.364116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:25.364116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:25.364116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:25.364116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:25.364116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:25.364116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:25.364116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:25.364116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:25.364116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:25.365084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:25.365084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:25.365084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:25.365084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:25.365084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:25.365084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:25.365084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:25.365084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:25.365084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:25.365084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:25.365084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:25.365084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:25.365084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 245/495] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:27:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:25.365084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb066e5-d06a-448a-904c-d71a3f957a83 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:25.365816 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb066e5-d06a-448a-904c-d71a3f957a83 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:25.365816 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb066e5-d06a-448a-904c-d71a3f957a83 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:25.368707 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 246/496] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:25.369571 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb066e5-d06a-448a-904c-d71a3f957a83 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:25.515767 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:25.521455 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73347cb-3bbe-4cc2-a8e6-7da5c4458d0a None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:25.523820 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb066e5-d06a-448a-904c-d71a3f957a83 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:25.558737 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74da927-e47e-4538-a186-9e0b6d3abbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:25.559124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2795da-3f19-4034-9726-23d0f972460c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:25.559408 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74da927-e47e-4538-a186-9e0b6d3abbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:25.559696 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74da927-e47e-4538-a186-9e0b6d3abbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:25.559974 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74da927-e47e-4538-a186-9e0b6d3abbb3 None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:25.560524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2795da-3f19-4034-9726-23d0f972460c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:25.560849 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74da927-e47e-4538-a186-9e0b6d3abbb3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:25.561168 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74da927-e47e-4538-a186-9e0b6d3abbb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:25.561369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2795da-3f19-4034-9726-23d0f972460c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:25.561806 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2795da-3f19-4034-9726-23d0f972460c None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:25.562621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74da927-e47e-4538-a186-9e0b6d3abbb3 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:25.562887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2795da-3f19-4034-9726-23d0f972460c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:25.563359 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2795da-3f19-4034-9726-23d0f972460c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:25.564916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2795da-3f19-4034-9726-23d0f972460c None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:25.581686 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 251/497] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:25.581857 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2795da-3f19-4034-9726-23d0f972460c None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:25.584692 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:25.603988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 247/498] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 23 22:27:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:25.603988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e3fcb-29b6-4771-b863-08570c28ecbf None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:25.603988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e3fcb-29b6-4771-b863-08570c28ecbf None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:25.603988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e3fcb-29b6-4771-b863-08570c28ecbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:25.620131 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-491e3fcb-29b6-4771-b863-08570c28ecbf None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:25.621452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-491e3fcb-29b6-4771-b863-08570c28ecbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:25.693837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2795da-3f19-4034-9726-23d0f972460c None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:25.704767 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 248/499] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:25.704767 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e3fcb-29b6-4771-b863-08570c28ecbf None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:25.736126 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3d3ae-616f-405a-899a-d9e6366cb2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:25.736815 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3d3ae-616f-405a-899a-d9e6366cb2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:25.737097 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3d3ae-616f-405a-899a-d9e6366cb2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:25.737371 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3d3ae-616f-405a-899a-d9e6366cb2c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:25.738116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3d3ae-616f-405a-899a-d9e6366cb2c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:25.738421 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3d3ae-616f-405a-899a-d9e6366cb2c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:25.739787 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3d3ae-616f-405a-899a-d9e6366cb2c9 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:25.748589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 252/500] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:27:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:25.749246 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3d3ae-616f-405a-899a-d9e6366cb2c9 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:25.786600 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:25.804188 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e3fcb-29b6-4771-b863-08570c28ecbf None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:25.812197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:25.842773 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df971e-0786-43fc-9e87-25b2e0fd161e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:25.842773 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df971e-0786-43fc-9e87-25b2e0fd161e None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:25.842773 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df971e-0786-43fc-9e87-25b2e0fd161e None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:25.842773 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df971e-0786-43fc-9e87-25b2e0fd161e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:25.846669 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df971e-0786-43fc-9e87-25b2e0fd161e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:25.848176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df971e-0786-43fc-9e87-25b2e0fd161e None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:25.850191 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3d3ae-616f-405a-899a-d9e6366cb2c9 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:25.864109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-64df971e-0786-43fc-9e87-25b2e0fd161e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 22:27:25.864109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:25.864109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:25.864109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:25.864109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:25.864109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:25.864109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:25.864109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:25.864109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:25.864109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:25.864109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:25.864109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:25.864109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:25.864109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:25.864109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:25.864109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:25.865215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:25.865215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:25.865215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:25.865215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:25.865215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:25.865215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:25.865215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:25.865215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:25.865215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:25.865215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:25.865215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:25.865215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:25.865215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:25.865215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:25.865215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:25.865215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:25.865215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:25.865215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:25.866123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:25.866123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:25.866123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:25.866123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:25.866123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:25.866123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:25.866123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:25.866123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:25.866123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 22:27:25.866123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:25.866123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 249/501] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:25.867190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaba635-3e0e-4152-a2bd-ae4bf7b56af6 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:25.867427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaba635-3e0e-4152-a2bd-ae4bf7b56af6 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:25.867813 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaba635-3e0e-4152-a2bd-ae4bf7b56af6 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:25.869392 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 250/502] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:25.869959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaba635-3e0e-4152-a2bd-ae4bf7b56af6 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:25.892732 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8822e-3a66-48e9-a6d4-4dcd9adb159a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:25.893382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8822e-3a66-48e9-a6d4-4dcd9adb159a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:25.893670 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8822e-3a66-48e9-a6d4-4dcd9adb159a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:25.893950 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8822e-3a66-48e9-a6d4-4dcd9adb159a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:25.960213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8822e-3a66-48e9-a6d4-4dcd9adb159a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:25.961714 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8822e-3a66-48e9-a6d4-4dcd9adb159a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:25.969734 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaba635-3e0e-4152-a2bd-ae4bf7b56af6 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:26.003213 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457c59b-085a-4703-b42d-587005c2e178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:26.004001 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457c59b-085a-4703-b42d-587005c2e178 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:26.004288 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457c59b-085a-4703-b42d-587005c2e178 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:26.004565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457c59b-085a-4703-b42d-587005c2e178 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:26.015179 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457c59b-085a-4703-b42d-587005c2e178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:26.022824 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:26.022967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-e457c59b-085a-4703-b42d-587005c2e178 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:26.022967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:26.022967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:26.022967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:26.022967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:26.022967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:26.022967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:26.022967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:26.022967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:26.022967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:26.022967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:26.022967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:26.022967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:26.022967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:26.022967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:26.022967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:26.023812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:26.023812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:26.023812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:26.023812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:26.023812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:26.023812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:26.023812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:26.023812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:26.023812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:26.023812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:26.023812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:26.023812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:26.023812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:26.023812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:26.023812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:26.023812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:26.024713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:26.024713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:26.024713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:26.024713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:26.024713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:26.024713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:26.024713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:26.024713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:26.024713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:26.024713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:26.024713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:26.024713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:26.024713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 251/503] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:27:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:26.025357 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457c59b-085a-4703-b42d-587005c2e178 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:26.030680 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:26.064213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 253/504] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:26.065130 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8822e-3a66-48e9-a6d4-4dcd9adb159a None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:26.128097 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457c59b-085a-4703-b42d-587005c2e178 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:26.161632 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43606a7a-18bc-489f-b835-77ce22d2528a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:26.161821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8822e-3a66-48e9-a6d4-4dcd9adb159a None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:26.162536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43606a7a-18bc-489f-b835-77ce22d2528a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:26.162821 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43606a7a-18bc-489f-b835-77ce22d2528a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:26.163103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43606a7a-18bc-489f-b835-77ce22d2528a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:26.174328 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43606a7a-18bc-489f-b835-77ce22d2528a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:26.175786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43606a7a-18bc-489f-b835-77ce22d2528a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:26.193911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-43606a7a-18bc-489f-b835-77ce22d2528a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:26.193911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:26.193911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:26.193911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:26.193911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:26.193911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:26.193911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:26.193911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:26.193911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:26.193911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:26.193911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:26.193911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:26.193911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:26.193911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:26.193911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:26.194761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:26.194761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:26.194761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:26.194761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:26.194761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:26.194761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:26.194761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:26.194761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:26.194761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:26.194761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:26.194761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:26.194761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:26.194761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:26.194761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:26.194761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:26.194761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:26.194761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:26.196270 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:26.196270 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:26.196270 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:26.196270 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:26.196270 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 252/505] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:26.197033 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a58f3-24de-42c1-a51c-407ef183df19 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:26.197330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a58f3-24de-42c1-a51c-407ef183df19 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:26.197517 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fc1ba-76e0-4e5f-90b5-3ecb62f83209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:26.197632 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a58f3-24de-42c1-a51c-407ef183df19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:26.198147 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fc1ba-76e0-4e5f-90b5-3ecb62f83209 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:26.198439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fc1ba-76e0-4e5f-90b5-3ecb62f83209 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:26.198718 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fc1ba-76e0-4e5f-90b5-3ecb62f83209 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:26.199560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fc1ba-76e0-4e5f-90b5-3ecb62f83209 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:26.199871 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fc1ba-76e0-4e5f-90b5-3ecb62f83209 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:26.201329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fc1ba-76e0-4e5f-90b5-3ecb62f83209 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:26.214230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-df2a58f3-24de-42c1-a51c-407ef183df19 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:26.215435 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-df2a58f3-24de-42c1-a51c-407ef183df19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:26.238891 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 254/506] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 23 22:27:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:26.239978 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fc1ba-76e0-4e5f-90b5-3ecb62f83209 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:26.263373 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:26.266213 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:26.285845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 253/507] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:26.285845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a58f3-24de-42c1-a51c-407ef183df19 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:26.348219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fc1ba-76e0-4e5f-90b5-3ecb62f83209 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:26.381745 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655c25b-9b9f-43d4-9f2d-831467af1372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:26.382834 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655c25b-9b9f-43d4-9f2d-831467af1372 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:26.383125 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655c25b-9b9f-43d4-9f2d-831467af1372 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:26.383452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655c25b-9b9f-43d4-9f2d-831467af1372 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:26.384210 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655c25b-9b9f-43d4-9f2d-831467af1372 None admin] RBAC: Adding query filter params (name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:26.384521 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655c25b-9b9f-43d4-9f2d-831467af1372 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:26.385939 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655c25b-9b9f-43d4-9f2d-831467af1372 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:26.389878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a58f3-24de-42c1-a51c-407ef183df19 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:26.393972 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 255/508] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:27:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:26.395011 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655c25b-9b9f-43d4-9f2d-831467af1372 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:26.425120 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5f2ed-a32c-463c-ac36-93e9b0657e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:26.425911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5f2ed-a32c-463c-ac36-93e9b0657e08 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:26.426201 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5f2ed-a32c-463c-ac36-93e9b0657e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:26.426479 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5f2ed-a32c-463c-ac36-93e9b0657e08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:26.436875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5f2ed-a32c-463c-ac36-93e9b0657e08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:26.438343 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5f2ed-a32c-463c-ac36-93e9b0657e08 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:26.455961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-b3e5f2ed-a32c-463c-ac36-93e9b0657e08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:26.455961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:26.455961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:26.455961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:26.455961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:26.455961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:26.455961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:26.455961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:26.455961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:26.455961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:26.455961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:26.455961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:26.455961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:26.455961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:26.455961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:26.456862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:26.456862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:26.456862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:26.456862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:26.456862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:26.456862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:26.456862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:26.456862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:26.456862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:26.456862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:26.456862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:26.456862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:26.456862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:26.456862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:26.456862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:26.456862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:26.456862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:26.457709 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:26.457709 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:26.457709 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:26.457709 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:26.457709 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 254/509] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:26.458121 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5f2ed-a32c-463c-ac36-93e9b0657e08 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:26.484286 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:26.493895 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:26.499735 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655c25b-9b9f-43d4-9f2d-831467af1372 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:26.532477 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231af76a-3866-4747-8815-1d1c6d9e0c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:26.533166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231af76a-3866-4747-8815-1d1c6d9e0c19 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:26.533454 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231af76a-3866-4747-8815-1d1c6d9e0c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:26.533732 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231af76a-3866-4747-8815-1d1c6d9e0c19 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:26.534460 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231af76a-3866-4747-8815-1d1c6d9e0c19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:26.534767 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231af76a-3866-4747-8815-1d1c6d9e0c19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:26.536168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231af76a-3866-4747-8815-1d1c6d9e0c19 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:26.548902 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 256/510] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:26.549701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231af76a-3866-4747-8815-1d1c6d9e0c19 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:26.560846 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5f2ed-a32c-463c-ac36-93e9b0657e08 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:26.593518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd3fcd-5e33-46d3-b73f-dcbd2e3d893d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:26.594158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd3fcd-5e33-46d3-b73f-dcbd2e3d893d None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:26.594446 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd3fcd-5e33-46d3-b73f-dcbd2e3d893d None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:26.594721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd3fcd-5e33-46d3-b73f-dcbd2e3d893d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:26.646696 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231af76a-3866-4747-8815-1d1c6d9e0c19 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:26.661819 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd3fcd-5e33-46d3-b73f-dcbd2e3d893d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:26.663284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd3fcd-5e33-46d3-b73f-dcbd2e3d893d None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:26.681010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc3089-f02b-4c7a-bff1-a67df639994b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:26.681798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc3089-f02b-4c7a-bff1-a67df639994b None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:26.682086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc3089-f02b-4c7a-bff1-a67df639994b None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:26.682366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc3089-f02b-4c7a-bff1-a67df639994b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:26.692248 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc3089-f02b-4c7a-bff1-a67df639994b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:26.693726 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc3089-f02b-4c7a-bff1-a67df639994b None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:26.696236 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:26.697614 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:26.711543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-b1dc3089-f02b-4c7a-bff1-a67df639994b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:26.711543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:26.711543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:26.711543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:26.711543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:26.711543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:26.711543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:26.711543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:26.711543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:26.711543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:26.711543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:26.711543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:26.711543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:26.711543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:26.711543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:26.712384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:26.712384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:26.712384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:26.712384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:26.712384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:26.712384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:26.712384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:26.712384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:26.712384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:26.712384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:26.712384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:26.712384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:26.712384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:26.712384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:26.712384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:26.712384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:26.712384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:26.713268 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:26.713268 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:26.713268 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:26.713268 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:26.716769 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 257/511] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:26.717801 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc3089-f02b-4c7a-bff1-a67df639994b None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:26.769892 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 255/512] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:26.769892 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd3fcd-5e33-46d3-b73f-dcbd2e3d893d None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:26.818161 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc3089-f02b-4c7a-bff1-a67df639994b None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:26.854495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78403f3e-65c3-447e-bd38-19f96bfde67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:26.855290 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78403f3e-65c3-447e-bd38-19f96bfde67f None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:26.855579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78403f3e-65c3-447e-bd38-19f96bfde67f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:26.855861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78403f3e-65c3-447e-bd38-19f96bfde67f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:26.867286 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78403f3e-65c3-447e-bd38-19f96bfde67f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:26.868561 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd3fcd-5e33-46d3-b73f-dcbd2e3d893d None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:26.868792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78403f3e-65c3-447e-bd38-19f96bfde67f None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:26.886666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-78403f3e-65c3-447e-bd38-19f96bfde67f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:26.886666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:26.886666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:26.886666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:26.886666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:26.886666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:26.886666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:26.886666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:26.886666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:26.886666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:26.886666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:26.886666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:26.886666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:26.886666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:26.886666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:26.888197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:26.888197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:26.888197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:26.888197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:26.888197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:26.888197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:26.888197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:26.888197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:26.888197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:26.888197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:26.888197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:26.888197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:26.888197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:26.888197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:26.888197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:26.888197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:26.888197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:26.889562 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:26.889562 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:26.889562 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:26.889562 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:26.889562 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 258/513] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:26.889562 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78403f3e-65c3-447e-bd38-19f96bfde67f None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:26.907771 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599fd11-6951-4d6e-a0ae-b60f4dc45b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:26.908351 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599fd11-6951-4d6e-a0ae-b60f4dc45b3e None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:26.908659 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599fd11-6951-4d6e-a0ae-b60f4dc45b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:26.908933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599fd11-6951-4d6e-a0ae-b60f4dc45b3e None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:26.909855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599fd11-6951-4d6e-a0ae-b60f4dc45b3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:26.910166 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599fd11-6951-4d6e-a0ae-b60f4dc45b3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:26.911583 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599fd11-6951-4d6e-a0ae-b60f4dc45b3e None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:26.914566 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:26.924409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 256/514] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:26.926039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599fd11-6951-4d6e-a0ae-b60f4dc45b3e None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:26.948644 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:27.010303 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78403f3e-65c3-447e-bd38-19f96bfde67f None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:27.030857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599fd11-6951-4d6e-a0ae-b60f4dc45b3e None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:27.051436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b9261-8234-4070-9186-1b8e651c741b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:27.052237 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b9261-8234-4070-9186-1b8e651c741b None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:27.052519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b9261-8234-4070-9186-1b8e651c741b None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:27.052821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b9261-8234-4070-9186-1b8e651c741b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:27.062373 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b9261-8234-4070-9186-1b8e651c741b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:27.063883 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b9261-8234-4070-9186-1b8e651c741b None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:27.068196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc246a-f5a6-4eb3-b6f7-1be062f35da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:27.070240 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc246a-f5a6-4eb3-b6f7-1be062f35da2 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:27.070870 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc246a-f5a6-4eb3-b6f7-1be062f35da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:27.071300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc246a-f5a6-4eb3-b6f7-1be062f35da2 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:27.072065 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc246a-f5a6-4eb3-b6f7-1be062f35da2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:27.072390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc246a-f5a6-4eb3-b6f7-1be062f35da2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:27.073846 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc246a-f5a6-4eb3-b6f7-1be062f35da2 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:27.079552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-062b9261-8234-4070-9186-1b8e651c741b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 22:27:27.079552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:27.079552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:27.079552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:27.079552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:27.079552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:27.079552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:27.079552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:27.079552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:27.079552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:27.079552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:27.079552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:27.079552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:27.079552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:27.079552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:27.079552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:27.080861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:27.080861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:27.080861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:27.080861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:27.080861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:27.080861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:27.080861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:27.080861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:27.080861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:27.080861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:27.080861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:27.080861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:27.080861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:27.080861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:27.080861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:27.080861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:27.080861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:27.080861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:27.081765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:27.081765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:27.081765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:27.081765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:27.081765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:27.081765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:27.081765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:27.081765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:27.081765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 22:27:27.081765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:27.081765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 259/515] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:27.081765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b9261-8234-4070-9186-1b8e651c741b None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:27.088296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 257/516] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:27.089263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc246a-f5a6-4eb3-b6f7-1be062f35da2 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:27.149747 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:27.196317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc246a-f5a6-4eb3-b6f7-1be062f35da2 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:27.198837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b9261-8234-4070-9186-1b8e651c741b None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:27.208677 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:27.233522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3843974-2953-40c5-a188-2dc81f91eb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:27.234210 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3843974-2953-40c5-a188-2dc81f91eb3c None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:27.234495 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3843974-2953-40c5-a188-2dc81f91eb3c None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:27.234699 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58fd0b-18ff-4020-8b9b-f8c1ef774e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:27.234811 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3843974-2953-40c5-a188-2dc81f91eb3c None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:27.235492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3843974-2953-40c5-a188-2dc81f91eb3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:27.235808 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3843974-2953-40c5-a188-2dc81f91eb3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:27.235957 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58fd0b-18ff-4020-8b9b-f8c1ef774e21 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:27.236203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58fd0b-18ff-4020-8b9b-f8c1ef774e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:27.236764 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58fd0b-18ff-4020-8b9b-f8c1ef774e21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:27.237273 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3843974-2953-40c5-a188-2dc81f91eb3c None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:27.249550 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 258/517] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:27.250360 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3843974-2953-40c5-a188-2dc81f91eb3c None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:27.254921 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58fd0b-18ff-4020-8b9b-f8c1ef774e21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:27.257516 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58fd0b-18ff-4020-8b9b-f8c1ef774e21 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:27.277514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-8b58fd0b-18ff-4020-8b9b-f8c1ef774e21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:27.277514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:27.277514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:27.277514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:27.277514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:27.277514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:27.277514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:27.277514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:27.277514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:27.277514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:27.277514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:27.277514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:27.277514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:27.277514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:27.277514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:27.278589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:27.278589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:27.278589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:27.278589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:27.278589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:27.278589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:27.278589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:27.278589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:27.278589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:27.278589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:27.278589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:27.278589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:27.278589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:27.278589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:27.278589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:27.278589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:27.278589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:27.279464 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:27.279464 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:27.279464 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:27.279464 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:27.279464 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 260/518] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:27.279848 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58fd0b-18ff-4020-8b9b-f8c1ef774e21 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:27.367910 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3843974-2953-40c5-a188-2dc81f91eb3c None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:27.387349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:27.389742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58fd0b-18ff-4020-8b9b-f8c1ef774e21 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:27.407899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4227c4-3635-40f4-b79c-2f91d11627e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:27.408728 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4227c4-3635-40f4-b79c-2f91d11627e5 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:27.409012 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4227c4-3635-40f4-b79c-2f91d11627e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:27.409309 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4227c4-3635-40f4-b79c-2f91d11627e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:27.417047 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4227c4-3635-40f4-b79c-2f91d11627e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:27.419052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4227c4-3635-40f4-b79c-2f91d11627e5 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:27.426994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c253ae-d470-485a-85aa-9446164c2857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:27.427671 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c253ae-d470-485a-85aa-9446164c2857 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:27.427964 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c253ae-d470-485a-85aa-9446164c2857 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:27.428248 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c253ae-d470-485a-85aa-9446164c2857 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:27.429137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c253ae-d470-485a-85aa-9446164c2857 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:27.429592 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c253ae-d470-485a-85aa-9446164c2857 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:27.431208 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c253ae-d470-485a-85aa-9446164c2857 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:27.433867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-7e4227c4-3635-40f4-b79c-2f91d11627e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:27.433867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:27.433867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:27.433867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:27.433867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:27.433867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:27.433867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:27.433867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:27.433867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:27.433867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:27.433867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:27.433867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:27.433867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:27.433867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:27.433867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:27.434901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:27.434901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:27.434901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:27.434901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:27.434901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:27.434901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:27.434901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:27.434901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:27.434901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:27.434901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:27.434901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:27.434901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:27.434901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:27.434901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:27.434901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:27.434901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:27.434901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:27.435756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:27.435756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:27.435756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:27.435756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:27.435756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:27.435756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:27.435756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:27.435756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:27.435756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 259/519] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:27.436562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4227c4-3635-40f4-b79c-2f91d11627e5 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:27.451199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 261/520] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 23 22:27:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:27.452505 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c253ae-d470-485a-85aa-9446164c2857 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:27.474771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:27.545086 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4227c4-3635-40f4-b79c-2f91d11627e5 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:27.561750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c253ae-d470-485a-85aa-9446164c2857 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:27.582594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220caf1-6b95-4571-819b-63f9914e7fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:27.583256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220caf1-6b95-4571-819b-63f9914e7fcb None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:27.583535 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220caf1-6b95-4571-819b-63f9914e7fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:27.583811 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220caf1-6b95-4571-819b-63f9914e7fcb None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:27.584512 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220caf1-6b95-4571-819b-63f9914e7fcb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:27.586482 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220caf1-6b95-4571-819b-63f9914e7fcb None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:27.598035 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5baa1a-2e74-4e17-8b1d-90cf436bca86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:27.599081 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5baa1a-2e74-4e17-8b1d-90cf436bca86 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:27.599581 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5baa1a-2e74-4e17-8b1d-90cf436bca86 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:27.600089 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5baa1a-2e74-4e17-8b1d-90cf436bca86 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:27.607645 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5baa1a-2e74-4e17-8b1d-90cf436bca86 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:27.609144 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5baa1a-2e74-4e17-8b1d-90cf436bca86 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:27.623919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-9c5baa1a-2e74-4e17-8b1d-90cf436bca86 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 22:27:27.623919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:27.623919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:27.623919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:27.623919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:27.623919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:27.623919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:27.623919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:27.623919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:27.623919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:27.623919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:27.623919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:27.623919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:27.623919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:27.623919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:27.623919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:27.625086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:27.625086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:27.625086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:27.625086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:27.625086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:27.625086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:27.625086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:27.625086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:27.625086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:27.625086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:27.625086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:27.625086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:27.625086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:27.625086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:27.625086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:27.625086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:27.625086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:27.625086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:27.626054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:27.626054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:27.626054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:27.626054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:27.626054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:27.626054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:27.626054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:27.626054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:27.626054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 22:27:27.626054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:27.626054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 262/521] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:27:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:27.626054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5baa1a-2e74-4e17-8b1d-90cf436bca86 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:27.698632 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 260/522] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 22:27:27 2022] POST /identity/v3/users => generated 247 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:27:27.699579 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220caf1-6b95-4571-819b-63f9914e7fcb None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:27.703847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:27.730351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5baa1a-2e74-4e17-8b1d-90cf436bca86 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:27.749641 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:27.772402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d92480-2be8-4b23-9866-12360ce985a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:27.773103 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d92480-2be8-4b23-9866-12360ce985a9 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:27.773493 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d92480-2be8-4b23-9866-12360ce985a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:27.773669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d92480-2be8-4b23-9866-12360ce985a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:27.774426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d92480-2be8-4b23-9866-12360ce985a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:27.774736 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d92480-2be8-4b23-9866-12360ce985a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:27.776166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d92480-2be8-4b23-9866-12360ce985a9 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:27.789910 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 263/523] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:27.791034 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d92480-2be8-4b23-9866-12360ce985a9 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:27.831716 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220caf1-6b95-4571-819b-63f9914e7fcb None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:27.871657 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e209d3-ca30-401a-8467-95308a625d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:27.872328 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e209d3-ca30-401a-8467-95308a625d27 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:27.872836 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e209d3-ca30-401a-8467-95308a625d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:27.873294 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e209d3-ca30-401a-8467-95308a625d27 None admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:27.874418 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e209d3-ca30-401a-8467-95308a625d27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:27.874418 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e209d3-ca30-401a-8467-95308a625d27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:27.875831 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e209d3-ca30-401a-8467-95308a625d27 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:27.897218 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 261/524] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:27.898678 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e209d3-ca30-401a-8467-95308a625d27 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:27.910623 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d92480-2be8-4b23-9866-12360ce985a9 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:27.949453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f96a9b-33df-4e7f-9aee-55f44746205a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:27.950231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f96a9b-33df-4e7f-9aee-55f44746205a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:27.950671 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f96a9b-33df-4e7f-9aee-55f44746205a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:27.951012 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f96a9b-33df-4e7f-9aee-55f44746205a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:27.961427 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:27.997100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:28.005836 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e209d3-ca30-401a-8467-95308a625d27 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:28.025559 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f96a9b-33df-4e7f-9aee-55f44746205a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:28.027165 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f96a9b-33df-4e7f-9aee-55f44746205a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:28.042627 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38420f-8a9a-41e2-be91-363128ceaa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:28.043410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38420f-8a9a-41e2-be91-363128ceaa25 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:28.043764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38420f-8a9a-41e2-be91-363128ceaa25 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:28.044113 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38420f-8a9a-41e2-be91-363128ceaa25 None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:28.045059 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38420f-8a9a-41e2-be91-363128ceaa25 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:28.045435 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38420f-8a9a-41e2-be91-363128ceaa25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:28.046933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38420f-8a9a-41e2-be91-363128ceaa25 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:28.069897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 262/525] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Jun 23 22:27:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:28.069897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38420f-8a9a-41e2-be91-363128ceaa25 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:28.135104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 264/526] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:28.136631 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f96a9b-33df-4e7f-9aee-55f44746205a None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:28.178476 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:28.182784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38420f-8a9a-41e2-be91-363128ceaa25 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:28.224539 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be1a4d-b413-4056-9491-d0a29b8ec43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:28.225283 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be1a4d-b413-4056-9491-d0a29b8ec43f None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:28.225571 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be1a4d-b413-4056-9491-d0a29b8ec43f None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:28.225849 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be1a4d-b413-4056-9491-d0a29b8ec43f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:28.233848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be1a4d-b413-4056-9491-d0a29b8ec43f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:28.235320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be1a4d-b413-4056-9491-d0a29b8ec43f None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:28.243156 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f96a9b-33df-4e7f-9aee-55f44746205a None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:28.249537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-26be1a4d-b413-4056-9491-d0a29b8ec43f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:28.249537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:28.249537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:28.249537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:28.249537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:28.249537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:28.249537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:28.249537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:28.249537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:28.249537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:28.249537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:28.249537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:28.249537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:28.249537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:28.249537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:28.250389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:28.250389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:28.250389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:28.250389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:28.250389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:28.250389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:28.250389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:28.250389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:28.250389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:28.250389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:28.250389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:28.250389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:28.250389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:28.250389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:28.250389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:28.250389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:28.250389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:28.251256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:28.251256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:28.251256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:28.251256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:28.251256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:28.251256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:28.251256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:28.251256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:28.251256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 263/527] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:28.252101 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be1a4d-b413-4056-9491-d0a29b8ec43f None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:28.269656 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:28.289212 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e8d47-29d1-407d-9308-71cb8aa8e731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:28.290312 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e8d47-29d1-407d-9308-71cb8aa8e731 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:28.290603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e8d47-29d1-407d-9308-71cb8aa8e731 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:28.290889 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e8d47-29d1-407d-9308-71cb8aa8e731 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:28.291734 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e8d47-29d1-407d-9308-71cb8aa8e731 None admin] RBAC: Adding query filter params (role.id=698ff821321a43d494fe8940cdd08564, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=159cc8befc3a4d57bbd313832084db62) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:28.292057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e8d47-29d1-407d-9308-71cb8aa8e731 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:28.293779 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e8d47-29d1-407d-9308-71cb8aa8e731 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:28.299898 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 265/528] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:27:28 2022] GET /identity/v3/role_assignments?user.id=159cc8befc3a4d57bbd313832084db62&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=698ff821321a43d494fe8940cdd08564 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:28.300986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e8d47-29d1-407d-9308-71cb8aa8e731 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:28.387780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be1a4d-b413-4056-9491-d0a29b8ec43f None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:28.436159 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342c44c-7f8b-4b15-84c0-7819f4f1947e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:28.437338 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342c44c-7f8b-4b15-84c0-7819f4f1947e None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:28.437970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342c44c-7f8b-4b15-84c0-7819f4f1947e None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:28.438458 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342c44c-7f8b-4b15-84c0-7819f4f1947e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:28.448225 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e8d47-29d1-407d-9308-71cb8aa8e731 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:28.450066 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:28.486724 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8dbdb-e6df-400d-874d-28b45b4a4430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:28.487899 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8dbdb-e6df-400d-874d-28b45b4a4430 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:28.488290 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8dbdb-e6df-400d-874d-28b45b4a4430 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:28.488722 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8dbdb-e6df-400d-874d-28b45b4a4430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:28.501568 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8dbdb-e6df-400d-874d-28b45b4a4430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:28.503040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8dbdb-e6df-400d-874d-28b45b4a4430 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:28.527735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342c44c-7f8b-4b15-84c0-7819f4f1947e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:28.529262 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342c44c-7f8b-4b15-84c0-7819f4f1947e None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:28.531370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-9cc8dbdb-e6df-400d-874d-28b45b4a4430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:28.531370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:28.531370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:28.531370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:28.531370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:28.531370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:28.531370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:28.531370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:28.531370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:28.531370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:28.531370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:28.531370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:28.531370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:28.531370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:28.531370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:28.532218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:28.532218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:28.532218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:28.532218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:28.532218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:28.532218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:28.532218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:28.532218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:28.532218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:28.532218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:28.532218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:28.532218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:28.532218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:28.532218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:28.532218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:28.532218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:28.532218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:28.533116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:28.533116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:28.533116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:28.533116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:28.539599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 266/529] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:28.539599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8dbdb-e6df-400d-874d-28b45b4a4430 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:28.546594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:28.639988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 264/530] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:28.641060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342c44c-7f8b-4b15-84c0-7819f4f1947e None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:28.678019 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8dbdb-e6df-400d-874d-28b45b4a4430 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:28.713675 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb0216-622f-4c16-a7a6-9323063412f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:28.714540 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb0216-622f-4c16-a7a6-9323063412f2 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:28.714904 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb0216-622f-4c16-a7a6-9323063412f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:28.715249 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb0216-622f-4c16-a7a6-9323063412f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:28.716124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb0216-622f-4c16-a7a6-9323063412f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:28.716527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb0216-622f-4c16-a7a6-9323063412f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:28.718101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb0216-622f-4c16-a7a6-9323063412f2 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:28.740992 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 267/531] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:28.742468 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb0216-622f-4c16-a7a6-9323063412f2 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:28.760865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:28.769290 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342c44c-7f8b-4b15-84c0-7819f4f1947e None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:28.792935 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:28.817954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965563d-8384-4f72-9ca0-277685624515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:28.820069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965563d-8384-4f72-9ca0-277685624515 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:28.820547 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965563d-8384-4f72-9ca0-277685624515 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:28.821015 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965563d-8384-4f72-9ca0-277685624515 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:28.821919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965563d-8384-4f72-9ca0-277685624515 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:28.822394 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965563d-8384-4f72-9ca0-277685624515 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:28.825606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965563d-8384-4f72-9ca0-277685624515 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:28.845020 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 265/532] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:28.846646 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965563d-8384-4f72-9ca0-277685624515 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:28.856841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb0216-622f-4c16-a7a6-9323063412f2 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:28.895024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba7152-ea29-4d91-a5d0-46418f439028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:28.895964 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba7152-ea29-4d91-a5d0-46418f439028 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:28.896431 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba7152-ea29-4d91-a5d0-46418f439028 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:28.896550 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba7152-ea29-4d91-a5d0-46418f439028 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:28.907904 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba7152-ea29-4d91-a5d0-46418f439028 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:28.909487 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba7152-ea29-4d91-a5d0-46418f439028 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:28.929171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-3cba7152-ea29-4d91-a5d0-46418f439028 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:28.929171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:28.929171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:28.929171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:28.929171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:28.929171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:28.929171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:28.929171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:28.929171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:28.929171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:28.929171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:28.929171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:28.929171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:28.929171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:28.929171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:28.930291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:28.930291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:28.930291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:28.930291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:28.930291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:28.930291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:28.930291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:28.930291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:28.930291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:28.930291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:28.930291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:28.930291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:28.930291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:28.930291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:28.930291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:28.930291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:28.930291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:28.931441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:28.931441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:28.931441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:28.931441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:28.931441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 268/533] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:28.931705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba7152-ea29-4d91-a5d0-46418f439028 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:28.970757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965563d-8384-4f72-9ca0-277685624515 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:29.001112 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:29.007921 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca439866-b4f7-4ffd-ae5c-f06b52872fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:29.008834 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca439866-b4f7-4ffd-ae5c-f06b52872fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:29.009284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca439866-b4f7-4ffd-ae5c-f06b52872fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:29.014315 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca439866-b4f7-4ffd-ae5c-f06b52872fa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:29.014315 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca439866-b4f7-4ffd-ae5c-f06b52872fa2 None admin] RBAC: Adding query filter params (role.id=698ff821321a43d494fe8940cdd08564, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=5d1911bfac984e058b0b11bb6400e782) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:29.014315 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca439866-b4f7-4ffd-ae5c-f06b52872fa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:29.014315 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca439866-b4f7-4ffd-ae5c-f06b52872fa2 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:29.024908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 266/534] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:27:28 2022] GET /identity/v3/role_assignments?user.id=5d1911bfac984e058b0b11bb6400e782&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=698ff821321a43d494fe8940cdd08564 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:29.026229 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca439866-b4f7-4ffd-ae5c-f06b52872fa2 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:29.040605 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba7152-ea29-4d91-a5d0-46418f439028 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:29.078754 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2aa2b-14b3-4e31-a0ff-06bee0ed12b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:29.079627 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2aa2b-14b3-4e31-a0ff-06bee0ed12b4 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:29.080159 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2aa2b-14b3-4e31-a0ff-06bee0ed12b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:29.080159 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2aa2b-14b3-4e31-a0ff-06bee0ed12b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:29.089025 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2aa2b-14b3-4e31-a0ff-06bee0ed12b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:29.090608 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2aa2b-14b3-4e31-a0ff-06bee0ed12b4 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:29.106669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-e1e2aa2b-14b3-4e31-a0ff-06bee0ed12b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:29.106669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:29.106669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:29.106669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:29.106669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:29.106669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:29.106669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:29.106669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:29.106669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:29.106669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:29.106669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:29.106669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:29.106669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:29.106669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:29.106669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:29.107615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:29.107615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:29.107615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:29.107615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:29.107615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:29.107615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:29.107615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:29.107615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:29.107615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:29.107615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:29.107615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:29.107615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:29.107615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:29.107615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:29.107615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:29.107615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:29.107615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:29.108453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:29.108453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:29.108453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:29.108453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:29.108453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:29.108453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:29.108453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:29.108453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:29.108453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 269/535] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:29.114169 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2aa2b-14b3-4e31-a0ff-06bee0ed12b4 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:29.124964 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:29.173151 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca439866-b4f7-4ffd-ae5c-f06b52872fa2 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:29.217389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00a3d0-0a6d-46f9-ba4a-d3796570bc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:29.218884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00a3d0-0a6d-46f9-ba4a-d3796570bc25 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:29.220105 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00a3d0-0a6d-46f9-ba4a-d3796570bc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:29.220627 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00a3d0-0a6d-46f9-ba4a-d3796570bc25 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:29.221606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00a3d0-0a6d-46f9-ba4a-d3796570bc25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:29.222127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00a3d0-0a6d-46f9-ba4a-d3796570bc25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:29.224279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00a3d0-0a6d-46f9-ba4a-d3796570bc25 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:29.240210 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 267/536] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:29.246501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00a3d0-0a6d-46f9-ba4a-d3796570bc25 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:29.279648 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2aa2b-14b3-4e31-a0ff-06bee0ed12b4 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:29.332098 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:29.365838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d55a8-8418-4292-beb2-2d39fde8ea84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:29.366557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d55a8-8418-4292-beb2-2d39fde8ea84 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:29.366854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d55a8-8418-4292-beb2-2d39fde8ea84 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:29.367178 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d55a8-8418-4292-beb2-2d39fde8ea84 None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:29.367942 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d55a8-8418-4292-beb2-2d39fde8ea84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:29.368258 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d55a8-8418-4292-beb2-2d39fde8ea84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:29.369752 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d55a8-8418-4292-beb2-2d39fde8ea84 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:29.388618 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:29.393044 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00a3d0-0a6d-46f9-ba4a-d3796570bc25 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:29.396531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 270/537] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:29.403271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d55a8-8418-4292-beb2-2d39fde8ea84 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:29.455870 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fd920-779a-4689-bc0a-a8117c53bb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:29.457177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fd920-779a-4689-bc0a-a8117c53bb39 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:29.457695 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fd920-779a-4689-bc0a-a8117c53bb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:29.458175 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fd920-779a-4689-bc0a-a8117c53bb39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:29.466873 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2fd920-779a-4689-bc0a-a8117c53bb39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:29.468761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2fd920-779a-4689-bc0a-a8117c53bb39 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:29.484806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-7f2fd920-779a-4689-bc0a-a8117c53bb39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:29.484806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:29.484806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:29.484806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:29.484806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:29.484806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:29.484806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:29.484806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:29.484806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:29.484806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:29.484806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:29.484806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:29.484806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:29.484806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:29.484806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:29.485712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:29.485712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:29.485712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:29.485712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:29.485712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:29.485712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:29.485712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:29.485712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:29.485712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:29.485712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:29.485712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:29.485712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:29.485712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:29.485712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:29.485712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:29.485712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:29.485712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:29.486574 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:29.486574 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:29.486574 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:29.486574 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:29.486574 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:29.486574 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:29.486574 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:29.486574 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:29.492826 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 268/538] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:29.496182 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fd920-779a-4689-bc0a-a8117c53bb39 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:29.520475 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d55a8-8418-4292-beb2-2d39fde8ea84 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:29.563708 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4aa94-64b0-4c36-be6e-5880d3b18e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:29.564294 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4aa94-64b0-4c36-be6e-5880d3b18e11 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:29.564603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4aa94-64b0-4c36-be6e-5880d3b18e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:29.564889 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4aa94-64b0-4c36-be6e-5880d3b18e11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:29.566171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b4aa94-64b0-4c36-be6e-5880d3b18e11 None admin] RBAC: Adding query filter params (role.id=698ff821321a43d494fe8940cdd08564, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=983801bb18924fe4b3150693765eb13b) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:29.566278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b4aa94-64b0-4c36-be6e-5880d3b18e11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:29.568556 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b4aa94-64b0-4c36-be6e-5880d3b18e11 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:29.575346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 271/539] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:27:29 2022] GET /identity/v3/role_assignments?user.id=983801bb18924fe4b3150693765eb13b&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=698ff821321a43d494fe8940cdd08564 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:27:29.690670 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fd920-779a-4689-bc0a-a8117c53bb39 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:29.726887 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9c44a-913b-4e6f-8fee-760632cc7997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:29.727770 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9c44a-913b-4e6f-8fee-760632cc7997 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:29.728059 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9c44a-913b-4e6f-8fee-760632cc7997 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:29.728338 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9c44a-913b-4e6f-8fee-760632cc7997 None admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:29.729175 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9c44a-913b-4e6f-8fee-760632cc7997 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:29.729498 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9c44a-913b-4e6f-8fee-760632cc7997 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:29.730963 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9c44a-913b-4e6f-8fee-760632cc7997 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:29.748407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 269/540] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:29.754542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4aa94-64b0-4c36-be6e-5880d3b18e11 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:29.870206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:29.875752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0eba42-0f4d-4adf-836f-0eea65b94ae8 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:29.876041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0eba42-0f4d-4adf-836f-0eea65b94ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:29.876319 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0eba42-0f4d-4adf-836f-0eea65b94ae8 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:29.878027 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 270/541] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:29.884280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68379b-5bec-446b-ae41-6a9242b98d4c None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:29.884568 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68379b-5bec-446b-ae41-6a9242b98d4c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:29.884882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68379b-5bec-446b-ae41-6a9242b98d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:29.902412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-bf68379b-5bec-446b-ae41-6a9242b98d4c None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:29.903721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-bf68379b-5bec-446b-ae41-6a9242b98d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:29.913883 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4aa94-64b0-4c36-be6e-5880d3b18e11 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:29.915326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:29.971993 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4c660-0a2a-4fab-82d1-2ede24400acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:29.972688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4c660-0a2a-4fab-82d1-2ede24400acd None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:29.972982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4c660-0a2a-4fab-82d1-2ede24400acd None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:29.973266 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4c660-0a2a-4fab-82d1-2ede24400acd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:29.974112 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b4c660-0a2a-4fab-82d1-2ede24400acd None admin] RBAC: Adding query filter params (role.id=698ff821321a43d494fe8940cdd08564, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=ec7412a9a3dc46369fe7b4771e3437de) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:29.974895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b4c660-0a2a-4fab-82d1-2ede24400acd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:29.982489 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b4c660-0a2a-4fab-82d1-2ede24400acd None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:29.994707 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 272/542] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:27:29 2022] GET /identity/v3/role_assignments?user.id=ec7412a9a3dc46369fe7b4771e3437de&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=698ff821321a43d494fe8940cdd08564 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:30.027794 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 271/543] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:30.065903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4c660-0a2a-4fab-82d1-2ede24400acd None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:30.192738 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4c660-0a2a-4fab-82d1-2ede24400acd None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:30.207967 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:30.231145 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a58c81-5519-496d-9820-0c7be0b146a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:30.232014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a58c81-5519-496d-9820-0c7be0b146a5 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:30.232302 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a58c81-5519-496d-9820-0c7be0b146a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:30.232609 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a58c81-5519-496d-9820-0c7be0b146a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:30.244214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a58c81-5519-496d-9820-0c7be0b146a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:30.252166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-d4a58c81-5519-496d-9820-0c7be0b146a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:30.252166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:30.252166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:30.252166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:30.252166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:30.252166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:30.252166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:30.252166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:30.252166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:30.252166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:30.252166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:30.252166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:30.252166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:30.252166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:30.252166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:30.252166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:30.253087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:30.253087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:30.253087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:30.253087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:30.253087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:30.253087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:30.253087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:30.253087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:30.253087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:30.253087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:30.253087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:30.253087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:30.253087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:30.253087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:30.253087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:30.253087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:30.253868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:30.253868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:30.253868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:30.253868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:30.253868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:30.253868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:30.253868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:30.253868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:30.253868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:30.253868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:30.253868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:30.253868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:30.253868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 273/544] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:27:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:30.261348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68379b-5bec-446b-ae41-6a9242b98d4c None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:30.425649 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68379b-5bec-446b-ae41-6a9242b98d4c None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:30.477409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71378a29-3a61-4b9e-a7e2-4e6436d9cb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:30.478022 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71378a29-3a61-4b9e-a7e2-4e6436d9cb5a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:30.478309 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71378a29-3a61-4b9e-a7e2-4e6436d9cb5a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:30.478592 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71378a29-3a61-4b9e-a7e2-4e6436d9cb5a None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:30.479357 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71378a29-3a61-4b9e-a7e2-4e6436d9cb5a None admin] RBAC: Adding query filter params (name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:30.479669 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71378a29-3a61-4b9e-a7e2-4e6436d9cb5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:30.481079 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71378a29-3a61-4b9e-a7e2-4e6436d9cb5a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:30.490050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 272/545] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:27:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:27:30.495992 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a58c81-5519-496d-9820-0c7be0b146a5 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:30.644003 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a58c81-5519-496d-9820-0c7be0b146a5 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:30.688018 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5adbbc-52c1-40c4-8ce1-bbbdc37aff1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:30.688851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5adbbc-52c1-40c4-8ce1-bbbdc37aff1d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:30.689137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5adbbc-52c1-40c4-8ce1-bbbdc37aff1d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:30.689418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5adbbc-52c1-40c4-8ce1-bbbdc37aff1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:30.700978 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5adbbc-52c1-40c4-8ce1-bbbdc37aff1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:30.702436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5adbbc-52c1-40c4-8ce1-bbbdc37aff1d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:30.729059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-0e5adbbc-52c1-40c4-8ce1-bbbdc37aff1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:30.729059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:30.729059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:30.729059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:30.729059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:30.729059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:30.729059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:30.729059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:30.729059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:30.729059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:30.729059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:30.729059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:30.729059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:30.729059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:30.729059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:30.729908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:30.729908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:30.729908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:30.729908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:30.729908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:30.729908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:30.729908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:30.729908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:30.729908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:30.729908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:30.729908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:30.729908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:30.729908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:30.729908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:30.729908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:30.729908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:30.729908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:30.730835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:30.730835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:30.730835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:30.730835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:30.730835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 274/546] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:30.736955 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71378a29-3a61-4b9e-a7e2-4e6436d9cb5a None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:30.744538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ca643-d6ad-4bef-ad7f-95adb1d391bc None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:30.744857 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ca643-d6ad-4bef-ad7f-95adb1d391bc None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:30.745142 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ca643-d6ad-4bef-ad7f-95adb1d391bc None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:30.746788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 275/547] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:30.748891 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:30.757218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d7346-c435-4f59-bc07-356313cda8ab None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:30.757513 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d7346-c435-4f59-bc07-356313cda8ab None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:30.757797 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d7346-c435-4f59-bc07-356313cda8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:30.782192 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-655d7346-c435-4f59-bc07-356313cda8ab None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:30.783414 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-655d7346-c435-4f59-bc07-356313cda8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:30.821782 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:30.852335 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71378a29-3a61-4b9e-a7e2-4e6436d9cb5a None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:30.892328 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99114bf8-7dc9-45a0-aaba-13ef3f41c7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:30.893477 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99114bf8-7dc9-45a0-aaba-13ef3f41c7d1 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:30.893979 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99114bf8-7dc9-45a0-aaba-13ef3f41c7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:30.894491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99114bf8-7dc9-45a0-aaba-13ef3f41c7d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:30.895585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99114bf8-7dc9-45a0-aaba-13ef3f41c7d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:30.896096 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99114bf8-7dc9-45a0-aaba-13ef3f41c7d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:30.897743 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99114bf8-7dc9-45a0-aaba-13ef3f41c7d1 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:30.908261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 276/548] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:27:30.910856 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 273/549] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:30.916369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d7346-c435-4f59-bc07-356313cda8ab None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:30.936162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99114bf8-7dc9-45a0-aaba-13ef3f41c7d1 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:31.009458 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:31.046662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99114bf8-7dc9-45a0-aaba-13ef3f41c7d1 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:31.084339 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e08538-5963-45dd-85ab-25ad1ffcd447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:31.085693 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e08538-5963-45dd-85ab-25ad1ffcd447 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:31.086188 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e08538-5963-45dd-85ab-25ad1ffcd447 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:31.086758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e08538-5963-45dd-85ab-25ad1ffcd447 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:31.097864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e08538-5963-45dd-85ab-25ad1ffcd447 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:31.105236 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d7346-c435-4f59-bc07-356313cda8ab None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:31.107112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-a7e08538-5963-45dd-85ab-25ad1ffcd447 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:31.107112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:31.107112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:31.107112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:31.107112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:31.107112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:31.107112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:31.107112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:31.107112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:31.107112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:31.107112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:31.107112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:31.107112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:31.107112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:31.107112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:31.107112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:31.108008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:31.108008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:31.108008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:31.108008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:31.108008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:31.108008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:31.108008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:31.108008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:31.108008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:31.108008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:31.108008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:31.108008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:31.108008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:31.108008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:31.108008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:31.108008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:31.108871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:31.108871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:31.108871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:31.108871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:31.108871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:31.108871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:31.108871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:31.108871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:31.108871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:31.108871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:31.108871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:31.108871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:31.111549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 274/550] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:27:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:31.118231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e08538-5963-45dd-85ab-25ad1ffcd447 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:31.186828 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96631800-67e0-4f8c-9797-31d88b320f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:31.187638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96631800-67e0-4f8c-9797-31d88b320f49 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:31.187927 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96631800-67e0-4f8c-9797-31d88b320f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:31.188205 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96631800-67e0-4f8c-9797-31d88b320f49 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:31.201356 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96631800-67e0-4f8c-9797-31d88b320f49 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:31.202832 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96631800-67e0-4f8c-9797-31d88b320f49 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:31.219374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-96631800-67e0-4f8c-9797-31d88b320f49 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:27:31.219374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:31.219374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:31.219374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:31.219374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:31.219374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:31.219374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:31.219374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:31.219374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:31.219374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:31.219374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:31.219374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:31.219374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:31.219374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:31.219374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:31.219374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:31.220523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:31.220523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:31.220523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:31.220523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:31.220523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:31.220523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:31.220523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:31.220523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:31.220523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:31.220523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:31.220523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:31.220523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:31.220523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:31.220523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:31.220523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:31.220523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:31.220523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:31.220523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:31.221469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:31.221469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:31.221469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:31.221469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:31.221469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:31.221469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:31.221469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:31.221469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:31.221469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:27:31.221469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:31.221469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 277/551] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:27:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:31.229982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96631800-67e0-4f8c-9797-31d88b320f49 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:31.233865 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e08538-5963-45dd-85ab-25ad1ffcd447 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:31.265301 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:31.272876 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:31.283682 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ab1ca-1a23-40c0-855b-e5b10d1ee591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:31.284626 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ab1ca-1a23-40c0-855b-e5b10d1ee591 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:31.285131 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ab1ca-1a23-40c0-855b-e5b10d1ee591 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:31.285595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ab1ca-1a23-40c0-855b-e5b10d1ee591 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:31.286637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ab1ca-1a23-40c0-855b-e5b10d1ee591 None admin] RBAC: Adding query filter params (name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:31.287096 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ab1ca-1a23-40c0-855b-e5b10d1ee591 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:31.289383 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ab1ca-1a23-40c0-855b-e5b10d1ee591 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:31.302469 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 275/552] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:27:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:31.311865 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ab1ca-1a23-40c0-855b-e5b10d1ee591 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:31.438815 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96631800-67e0-4f8c-9797-31d88b320f49 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:31.482831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d0f7b-17a9-4378-976f-43ebc9d4dbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:31.483895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d0f7b-17a9-4378-976f-43ebc9d4dbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:31.484419 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d0f7b-17a9-4378-976f-43ebc9d4dbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:31.484876 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d0f7b-17a9-4378-976f-43ebc9d4dbf3 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:31.485873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d0f7b-17a9-4378-976f-43ebc9d4dbf3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:31.486334 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d0f7b-17a9-4378-976f-43ebc9d4dbf3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:31.487933 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d0f7b-17a9-4378-976f-43ebc9d4dbf3 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:31.508206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 278/553] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 23 22:27:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:31.510846 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ab1ca-1a23-40c0-855b-e5b10d1ee591 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:31.513268 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d0f7b-17a9-4378-976f-43ebc9d4dbf3 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:31.578558 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a987091-f881-40c3-bdf8-7362746a0942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:31.579654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a987091-f881-40c3-bdf8-7362746a0942 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:31.580145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a987091-f881-40c3-bdf8-7362746a0942 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:31.580644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a987091-f881-40c3-bdf8-7362746a0942 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:31.594239 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a987091-f881-40c3-bdf8-7362746a0942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:31.596113 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a987091-f881-40c3-bdf8-7362746a0942 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:31.615936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-0a987091-f881-40c3-bdf8-7362746a0942 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:31.615936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:31.615936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:31.615936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:31.615936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:31.615936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:31.615936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:31.615936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:31.615936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:31.615936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:31.615936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:31.615936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:31.615936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:31.615936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:31.615936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:31.617744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:31.617744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:31.617744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:31.617744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:31.617744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:31.617744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:31.617744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:31.617744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:31.617744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:31.617744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:31.617744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:31.617744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:31.617744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:31.617744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:31.617744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:31.617744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:31.617744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:31.618721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:31.618721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:31.618721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:31.618721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:31.621100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 276/554] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:31.624528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d845b0-3658-421a-ac6a-aefee98aec30 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:31.624861 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d845b0-3658-421a-ac6a-aefee98aec30 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:31.625147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d845b0-3658-421a-ac6a-aefee98aec30 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:31.630264 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 277/555] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:27:31.631024 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d845b0-3658-421a-ac6a-aefee98aec30 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:31.641495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:31.672431 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d0f7b-17a9-4378-976f-43ebc9d4dbf3 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:31.687892 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:31.709601 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa2be7-aa53-4c06-89d7-2a24ab039e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:31.710293 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa2be7-aa53-4c06-89d7-2a24ab039e7e None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:31.710585 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa2be7-aa53-4c06-89d7-2a24ab039e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:31.710851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa2be7-aa53-4c06-89d7-2a24ab039e7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:31.759525 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d845b0-3658-421a-ac6a-aefee98aec30 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:31.796976 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc1a37-21b1-406a-8653-cfc158031d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:31.797644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc1a37-21b1-406a-8653-cfc158031d06 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:31.797931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc1a37-21b1-406a-8653-cfc158031d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:31.798213 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc1a37-21b1-406a-8653-cfc158031d06 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:31.799049 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc1a37-21b1-406a-8653-cfc158031d06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:31.799371 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc1a37-21b1-406a-8653-cfc158031d06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:31.800784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc1a37-21b1-406a-8653-cfc158031d06 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:31.801716 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa2be7-aa53-4c06-89d7-2a24ab039e7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:31.804934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa2be7-aa53-4c06-89d7-2a24ab039e7e None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:31.813459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 278/556] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:31.814839 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc921f26-e1b7-447f-a62f-c2737f2cf8f0 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:31.815444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc921f26-e1b7-447f-a62f-c2737f2cf8f0 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:31.815555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc921f26-e1b7-447f-a62f-c2737f2cf8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:31.833446 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-cc921f26-e1b7-447f-a62f-c2737f2cf8f0 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:31.834668 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-cc921f26-e1b7-447f-a62f-c2737f2cf8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:31.900780 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:31.916831 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 279/557] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:31.918269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc921f26-e1b7-447f-a62f-c2737f2cf8f0 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:31.934227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 279/558] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 22:27:31.941791 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa2be7-aa53-4c06-89d7-2a24ab039e7e None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:31.966764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:32.048252 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc921f26-e1b7-447f-a62f-c2737f2cf8f0 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:32.067105 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa2be7-aa53-4c06-89d7-2a24ab039e7e None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:32.093216 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3a438-aac6-45af-ae4f-4bba06b389be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:32.094107 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3a438-aac6-45af-ae4f-4bba06b389be None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:32.094401 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3a438-aac6-45af-ae4f-4bba06b389be None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:32.094683 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3a438-aac6-45af-ae4f-4bba06b389be None admin] PATH_INFO: `/v3/users/nova` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:32.105555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c3a438-aac6-45af-ae4f-4bba06b389be None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:32.107251 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c3a438-aac6-45af-ae4f-4bba06b389be None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:32.108053 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e80b704-e6d9-417d-b0a6-b7eba76a31cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:32.109354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80b704-e6d9-417d-b0a6-b7eba76a31cd None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:32.109845 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80b704-e6d9-417d-b0a6-b7eba76a31cd None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:32.110371 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80b704-e6d9-417d-b0a6-b7eba76a31cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:32.121676 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e80b704-e6d9-417d-b0a6-b7eba76a31cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:32.123428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e80b704-e6d9-417d-b0a6-b7eba76a31cd None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:32.125466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-67c3a438-aac6-45af-ae4f-4bba06b389be None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:27:32.125466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:32.125466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:32.125466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:32.125466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:32.125466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:32.125466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:32.125466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:32.125466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:32.125466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:32.125466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:32.125466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:32.125466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:32.125466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:32.125466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:32.125466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:32.126388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:32.126388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:32.126388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:32.126388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:32.126388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:32.126388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:32.126388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:32.126388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:32.126388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:32.126388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:32.126388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:32.126388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:32.126388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:32.126388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:32.126388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:32.126388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:32.126388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:32.126388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:32.127332 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:32.127332 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:32.127332 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:32.127332 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:32.127332 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:32.127332 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:32.127332 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:32.127332 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:32.127332 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:27:32.127332 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:32.127332 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 280/559] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 23 22:27:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:32.128403 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3a438-aac6-45af-ae4f-4bba06b389be None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:32.147250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-0e80b704-e6d9-417d-b0a6-b7eba76a31cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:32.147250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:32.147250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:32.147250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:32.147250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:32.147250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:32.147250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:32.147250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:32.147250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:32.147250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:32.147250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:32.147250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:32.147250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:32.147250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:32.147250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:32.149079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:32.149079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:32.149079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:32.149079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:32.149079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:32.149079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:32.149079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:32.149079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:32.149079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:32.149079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:32.149079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:32.149079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:32.149079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:32.149079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:32.149079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:32.149079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:32.149079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:32.150014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:32.150014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:32.150014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:32.150014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:32.156799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 280/560] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:27:32.158160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e80b704-e6d9-417d-b0a6-b7eba76a31cd None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:32.191444 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:32.242942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:32.254142 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3a438-aac6-45af-ae4f-4bba06b389be None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:32.293839 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011531f9-b54c-4a5a-bb39-d22b4912fc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:32.294384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e80b704-e6d9-417d-b0a6-b7eba76a31cd None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:32.295422 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011531f9-b54c-4a5a-bb39-d22b4912fc57 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:32.295986 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011531f9-b54c-4a5a-bb39-d22b4912fc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:32.296444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011531f9-b54c-4a5a-bb39-d22b4912fc57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:32.308406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011531f9-b54c-4a5a-bb39-d22b4912fc57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:32.317349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-011531f9-b54c-4a5a-bb39-d22b4912fc57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:32.317349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:32.317349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:32.317349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:32.317349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:32.317349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:32.317349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:32.317349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:32.317349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:32.317349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:32.317349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:32.317349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:32.317349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:32.317349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:32.317349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:32.317349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:32.318324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:32.318324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:32.318324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:32.318324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:32.318324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:32.318324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:32.318324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:32.318324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:32.318324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:32.318324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:32.318324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:32.318324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:32.318324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:32.318324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:32.318324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:32.318324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:32.319276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:32.319276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:32.319276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:32.319276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:32.319276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:32.319276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:32.319276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:32.319276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:32.319276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:32.319276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:32.319276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:32.319276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:32.321897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 281/561] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:27:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:32.323117 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011531f9-b54c-4a5a-bb39-d22b4912fc57 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:32.333792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b578d1-7e3c-45bc-9f0d-95a5b1615c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:32.334465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b578d1-7e3c-45bc-9f0d-95a5b1615c6a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:32.335161 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b578d1-7e3c-45bc-9f0d-95a5b1615c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:32.336032 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b578d1-7e3c-45bc-9f0d-95a5b1615c6a None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:32.337152 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b578d1-7e3c-45bc-9f0d-95a5b1615c6a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:32.337470 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b578d1-7e3c-45bc-9f0d-95a5b1615c6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:32.338996 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b578d1-7e3c-45bc-9f0d-95a5b1615c6a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:32.409326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 281/562] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 23 22:27:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:32.410696 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58e0dd-1f8a-4feb-af2b-75378e9044d3 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:32.411062 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58e0dd-1f8a-4feb-af2b-75378e9044d3 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:32.411419 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58e0dd-1f8a-4feb-af2b-75378e9044d3 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:32.412940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 282/563] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:32.414193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58e0dd-1f8a-4feb-af2b-75378e9044d3 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:32.478961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011531f9-b54c-4a5a-bb39-d22b4912fc57 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:32.499623 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:32.512072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84477498-f851-4452-b48d-aa4637366567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:32.512769 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84477498-f851-4452-b48d-aa4637366567 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:32.513055 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84477498-f851-4452-b48d-aa4637366567 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:32.513331 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84477498-f851-4452-b48d-aa4637366567 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:32.514056 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84477498-f851-4452-b48d-aa4637366567 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:32.514412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84477498-f851-4452-b48d-aa4637366567 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:32.515832 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84477498-f851-4452-b48d-aa4637366567 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:32.516511 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58e0dd-1f8a-4feb-af2b-75378e9044d3 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:32.528270 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 282/564] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:32.529102 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84477498-f851-4452-b48d-aa4637366567 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:32.541023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:32.552072 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84414b-53fb-4881-80fb-fe64afac3cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:32.552770 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84414b-53fb-4881-80fb-fe64afac3cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:32.553060 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84414b-53fb-4881-80fb-fe64afac3cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:32.553344 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84414b-53fb-4881-80fb-fe64afac3cd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:32.554241 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84414b-53fb-4881-80fb-fe64afac3cd5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:32.554553 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84414b-53fb-4881-80fb-fe64afac3cd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:32.555918 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84414b-53fb-4881-80fb-fe64afac3cd5 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:32.563932 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 283/565] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:27:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:32.565157 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8d3a2-29e8-44ea-bee0-cd449bd1b11c None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:32.565450 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8d3a2-29e8-44ea-bee0-cd449bd1b11c None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:32.565737 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8d3a2-29e8-44ea-bee0-cd449bd1b11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:32.582624 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-d3e8d3a2-29e8-44ea-bee0-cd449bd1b11c None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:32.583828 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d3e8d3a2-29e8-44ea-bee0-cd449bd1b11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:32.632998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84477498-f851-4452-b48d-aa4637366567 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:32.653593 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 284/566] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:32.654908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216395d-927e-4ead-93bb-0cf60ab18c50 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:32.655202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216395d-927e-4ead-93bb-0cf60ab18c50 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:32.655480 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216395d-927e-4ead-93bb-0cf60ab18c50 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:32.657026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 285/567] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:32.657731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216395d-927e-4ead-93bb-0cf60ab18c50 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:32.671367 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed55674-956c-41f7-b8ae-7e4bf87caa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:32.672412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed55674-956c-41f7-b8ae-7e4bf87caa77 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:32.672936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed55674-956c-41f7-b8ae-7e4bf87caa77 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:32.673308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed55674-956c-41f7-b8ae-7e4bf87caa77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:32.684284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed55674-956c-41f7-b8ae-7e4bf87caa77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:32.685798 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed55674-956c-41f7-b8ae-7e4bf87caa77 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:32.703804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-8ed55674-956c-41f7-b8ae-7e4bf87caa77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:32.703804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:32.703804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:32.703804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:32.703804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:32.703804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:32.703804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:32.703804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:32.703804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:32.703804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:32.703804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:32.703804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:32.703804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:32.703804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:32.703804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:32.704844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:32.704844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:32.704844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:32.704844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:32.704844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:32.704844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:32.704844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:32.704844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:32.704844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:32.704844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:32.704844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:32.704844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:32.704844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:32.704844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:32.704844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:32.704844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:32.704844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:32.706051 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:32.706051 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:32.706051 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:32.706051 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:32.706051 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 283/568] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:32.706051 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed55674-956c-41f7-b8ae-7e4bf87caa77 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:32.739842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:32.764116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216395d-927e-4ead-93bb-0cf60ab18c50 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:32.766445 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:32.797759 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8dcf0-108b-4aa1-a17c-a6a3215282cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:32.798639 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8dcf0-108b-4aa1-a17c-a6a3215282cd None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:32.798992 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8dcf0-108b-4aa1-a17c-a6a3215282cd None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:32.799278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8dcf0-108b-4aa1-a17c-a6a3215282cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:32.804568 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed55674-956c-41f7-b8ae-7e4bf87caa77 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:32.806626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec8dcf0-108b-4aa1-a17c-a6a3215282cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:32.808063 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec8dcf0-108b-4aa1-a17c-a6a3215282cd None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:32.822440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-3ec8dcf0-108b-4aa1-a17c-a6a3215282cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:32.822440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:32.822440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:32.822440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:32.822440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:32.822440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:32.822440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:32.822440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:32.822440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:32.822440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:32.822440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:32.822440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:32.822440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:32.822440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:32.822440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:32.823520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:32.823520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:32.823520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:32.823520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:32.823520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:32.823520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:32.823520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:32.823520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:32.823520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:32.823520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:32.823520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:32.823520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:32.823520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:32.823520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:32.823520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:32.823520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:32.823520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:32.824410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:32.824410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:32.824410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:32.824410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:32.824410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:32.824410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:32.824410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:32.824410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:32.824410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 286/569] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:32.824951 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1498b8b-7f38-405c-bc57-380695371aa3 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:32.825033 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1498b8b-7f38-405c-bc57-380695371aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:32.825342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1498b8b-7f38-405c-bc57-380695371aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:32.841193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-a1498b8b-7f38-405c-bc57-380695371aa3 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:32.842381 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a1498b8b-7f38-405c-bc57-380695371aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:32.842487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee42a33-12bc-47a6-bcca-3025599de144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:32.843277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee42a33-12bc-47a6-bcca-3025599de144 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:32.843565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee42a33-12bc-47a6-bcca-3025599de144 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:32.843844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee42a33-12bc-47a6-bcca-3025599de144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:32.854704 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee42a33-12bc-47a6-bcca-3025599de144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:32.856157 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee42a33-12bc-47a6-bcca-3025599de144 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:32.873534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-0ee42a33-12bc-47a6-bcca-3025599de144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:32.873534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:32.873534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:32.873534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:32.873534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:32.873534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:32.873534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:32.873534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:32.873534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:32.873534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:32.873534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:32.873534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:32.873534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:32.873534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:32.873534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:32.874585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:32.874585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:32.874585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:32.874585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:32.874585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:32.874585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:32.874585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:32.874585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:32.874585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:32.874585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:32.874585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:32.874585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:32.874585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:32.874585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:32.874585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:32.874585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:32.874585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:32.880325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:32.880325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:32.880325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:32.880325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:32.880325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 284/570] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:32.880325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee42a33-12bc-47a6-bcca-3025599de144 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:32.917180 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 287/571] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:32.917180 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1498b8b-7f38-405c-bc57-380695371aa3 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:32.975247 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee42a33-12bc-47a6-bcca-3025599de144 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:32.981403 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:32.988806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:33.008924 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45e9b9-fc30-430b-99fd-05a9fd09e19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:33.009716 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45e9b9-fc30-430b-99fd-05a9fd09e19f None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:33.010001 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45e9b9-fc30-430b-99fd-05a9fd09e19f None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:33.010276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45e9b9-fc30-430b-99fd-05a9fd09e19f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:33.015363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1498b8b-7f38-405c-bc57-380695371aa3 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:33.020627 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45e9b9-fc30-430b-99fd-05a9fd09e19f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:33.028671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-ff45e9b9-fc30-430b-99fd-05a9fd09e19f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:33.028671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:33.028671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:33.028671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:33.028671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:33.028671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:33.028671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:33.028671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:33.028671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:33.028671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:33.028671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:33.028671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:33.028671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:33.028671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:33.028671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:33.028671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:33.029790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:33.029790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:33.029790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:33.029790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:33.029790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:33.029790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:33.029790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:33.029790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:33.029790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:33.029790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:33.029790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:33.029790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:33.029790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:33.029790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:33.029790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:33.029790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:33.030844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:33.030844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:33.030844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:33.030844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:33.030844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:33.030844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:33.030844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:33.030844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:33.030844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:33.030844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:33.030844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:33.030844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:33.030844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 285/572] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:27:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:33.031669 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45e9b9-fc30-430b-99fd-05a9fd09e19f None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:33.055601 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a5a85-7c21-4665-8ccf-11f2d86d1a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:33.056258 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a5a85-7c21-4665-8ccf-11f2d86d1a2e None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:33.056539 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a5a85-7c21-4665-8ccf-11f2d86d1a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:33.056849 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a5a85-7c21-4665-8ccf-11f2d86d1a2e None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:33.057569 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a5a85-7c21-4665-8ccf-11f2d86d1a2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:33.057875 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a5a85-7c21-4665-8ccf-11f2d86d1a2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:33.059241 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a5a85-7c21-4665-8ccf-11f2d86d1a2e None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:33.070338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 288/573] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:33.071123 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a5a85-7c21-4665-8ccf-11f2d86d1a2e None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:33.130937 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45e9b9-fc30-430b-99fd-05a9fd09e19f None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:33.164682 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e5864-0c6e-43c3-aa59-6f2d6cd3b282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:33.165460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e5864-0c6e-43c3-aa59-6f2d6cd3b282 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:33.165745 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e5864-0c6e-43c3-aa59-6f2d6cd3b282 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:33.166026 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e5864-0c6e-43c3-aa59-6f2d6cd3b282 None admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:33.166814 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e5864-0c6e-43c3-aa59-6f2d6cd3b282 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:33.167371 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e5864-0c6e-43c3-aa59-6f2d6cd3b282 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:33.168822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e5864-0c6e-43c3-aa59-6f2d6cd3b282 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:33.172731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a5a85-7c21-4665-8ccf-11f2d86d1a2e None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:33.180895 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 286/574] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:33.181772 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e5864-0c6e-43c3-aa59-6f2d6cd3b282 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:33.201816 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:33.210910 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04222de3-3cc6-4907-82ac-b8a4b20823fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:33.213323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04222de3-3cc6-4907-82ac-b8a4b20823fe None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:33.213323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04222de3-3cc6-4907-82ac-b8a4b20823fe None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:33.213323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04222de3-3cc6-4907-82ac-b8a4b20823fe None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:33.213323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04222de3-3cc6-4907-82ac-b8a4b20823fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:33.213323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04222de3-3cc6-4907-82ac-b8a4b20823fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:33.214592 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04222de3-3cc6-4907-82ac-b8a4b20823fe None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:33.215825 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:33.226508 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 289/575] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:33.227186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04222de3-3cc6-4907-82ac-b8a4b20823fe None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:33.281696 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e5864-0c6e-43c3-aa59-6f2d6cd3b282 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:33.315718 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794a920-b608-4001-82bc-e6e947ea0425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:33.316524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794a920-b608-4001-82bc-e6e947ea0425 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:33.316832 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794a920-b608-4001-82bc-e6e947ea0425 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:33.317110 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794a920-b608-4001-82bc-e6e947ea0425 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:33.327406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794a920-b608-4001-82bc-e6e947ea0425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:33.330809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04222de3-3cc6-4907-82ac-b8a4b20823fe None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:33.335397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-1794a920-b608-4001-82bc-e6e947ea0425 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:33.335397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:33.335397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:33.335397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:33.335397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:33.335397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:33.335397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:33.335397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:33.335397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:33.335397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:33.335397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:33.335397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:33.335397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:33.335397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:33.335397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:33.335397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:33.336346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:33.336346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:33.336346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:33.336346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:33.336346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:33.336346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:33.336346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:33.336346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:33.336346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:33.336346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:33.336346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:33.336346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:33.336346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:33.336346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:33.336346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:33.336346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:33.337217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:33.337217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:33.337217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:33.337217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:33.337217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:33.337217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:33.337217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:33.337217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:33.337217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:33.337217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:33.337217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:33.337217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:33.337217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 287/576] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:27:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:33.337900 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794a920-b608-4001-82bc-e6e947ea0425 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:33.365351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2febb-6914-43dd-b2bb-1220b5d05e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:33.366017 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2febb-6914-43dd-b2bb-1220b5d05e93 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:33.366993 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2febb-6914-43dd-b2bb-1220b5d05e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:33.367436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2febb-6914-43dd-b2bb-1220b5d05e93 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:33.368344 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2febb-6914-43dd-b2bb-1220b5d05e93 None admin] RBAC: Adding query filter params (name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:33.368694 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2febb-6914-43dd-b2bb-1220b5d05e93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:33.370064 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2febb-6914-43dd-b2bb-1220b5d05e93 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:33.377871 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 290/577] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:27:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:33.378647 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2febb-6914-43dd-b2bb-1220b5d05e93 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:33.443745 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:33.457884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:33.464594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794a920-b608-4001-82bc-e6e947ea0425 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:33.493970 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2febb-6914-43dd-b2bb-1220b5d05e93 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:33.499615 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d6f3c-2b83-4be2-9a0f-850f5ec3338b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:33.499615 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d6f3c-2b83-4be2-9a0f-850f5ec3338b None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:33.499615 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d6f3c-2b83-4be2-9a0f-850f5ec3338b None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:33.499917 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d6f3c-2b83-4be2-9a0f-850f5ec3338b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:33.506450 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d6f3c-2b83-4be2-9a0f-850f5ec3338b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:33.508402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d6f3c-2b83-4be2-9a0f-850f5ec3338b None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:33.523043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-598d6f3c-2b83-4be2-9a0f-850f5ec3338b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:33.523043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:33.523043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:33.523043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:33.523043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:33.523043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:33.523043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:33.523043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:33.523043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:33.523043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:33.523043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:33.523043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:33.523043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:33.523043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:33.523043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:33.523988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:33.523988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:33.523988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:33.523988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:33.523988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:33.523988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:33.523988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:33.523988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:33.523988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:33.523988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:33.523988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:33.523988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:33.523988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:33.523988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:33.523988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:33.523988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:33.523988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:33.524918 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:33.524918 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:33.524918 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:33.524918 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:33.524918 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:33.524918 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:33.524918 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:33.524918 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:33.524918 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 288/578] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:33.525908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d6f3c-2b83-4be2-9a0f-850f5ec3338b None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:33.530401 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e762c-7791-4612-b425-e9d1f7bf0eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:33.531059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e762c-7791-4612-b425-e9d1f7bf0eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:33.531363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e762c-7791-4612-b425-e9d1f7bf0eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:33.531636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e762c-7791-4612-b425-e9d1f7bf0eb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:33.532466 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e762c-7791-4612-b425-e9d1f7bf0eb8 None admin] RBAC: Adding query filter params (role.id=698ff821321a43d494fe8940cdd08564, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=b321586cdb3d4d25904472aaa4adba71) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:33.532802 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e762c-7791-4612-b425-e9d1f7bf0eb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:33.534225 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e762c-7791-4612-b425-e9d1f7bf0eb8 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:33.540799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 291/579] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:27:33 2022] GET /identity/v3/role_assignments?user.id=b321586cdb3d4d25904472aaa4adba71&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=698ff821321a43d494fe8940cdd08564 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:33.541582 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e762c-7791-4612-b425-e9d1f7bf0eb8 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:33.636456 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d6f3c-2b83-4be2-9a0f-850f5ec3338b None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:33.665303 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e762c-7791-4612-b425-e9d1f7bf0eb8 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:33.674405 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920c919-1402-4408-9bb6-3056d701c519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:33.675228 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920c919-1402-4408-9bb6-3056d701c519 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:33.675516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920c919-1402-4408-9bb6-3056d701c519 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:33.675805 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920c919-1402-4408-9bb6-3056d701c519 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:33.680742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:33.684738 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920c919-1402-4408-9bb6-3056d701c519 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:33.686209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920c919-1402-4408-9bb6-3056d701c519 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:33.701320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-5920c919-1402-4408-9bb6-3056d701c519 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 22:27:33.701320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:33.701320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:33.701320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:33.701320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:33.701320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:33.701320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:33.701320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:33.701320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:33.701320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:33.701320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:33.701320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:33.701320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:33.701320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:33.701320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:33.701320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:33.702452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:33.702452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:33.702452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:33.702452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:33.702452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:33.702452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:33.702452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:33.702452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:33.702452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:33.702452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:33.702452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:33.702452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:33.702452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:33.702452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:33.702452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:33.702452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:33.702452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:33.702452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:33.703667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:33.703667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:33.703667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:33.703667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:33.703667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:33.703667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:33.703667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:33.703667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:33.703667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 22:27:33.703667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:33.703667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 289/580] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:33.703667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920c919-1402-4408-9bb6-3056d701c519 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:33.707145 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256fca9-0d6c-46a4-8873-420d4cc6c216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:33.707896 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256fca9-0d6c-46a4-8873-420d4cc6c216 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:33.708191 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256fca9-0d6c-46a4-8873-420d4cc6c216 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:33.708722 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256fca9-0d6c-46a4-8873-420d4cc6c216 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:33.709258 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256fca9-0d6c-46a4-8873-420d4cc6c216 None admin] RBAC: Adding query filter params (name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:33.710781 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256fca9-0d6c-46a4-8873-420d4cc6c216 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:33.711887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256fca9-0d6c-46a4-8873-420d4cc6c216 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:33.721542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 292/581] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:27:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:33.722368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256fca9-0d6c-46a4-8873-420d4cc6c216 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:33.725164 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:33.822307 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920c919-1402-4408-9bb6-3056d701c519 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:33.831052 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256fca9-0d6c-46a4-8873-420d4cc6c216 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:33.860910 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e939b0-5ed0-470f-a860-a24ebc3b0fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:33.861702 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e939b0-5ed0-470f-a860-a24ebc3b0fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:33.861987 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e939b0-5ed0-470f-a860-a24ebc3b0fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:33.862269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e939b0-5ed0-470f-a860-a24ebc3b0fa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:33.868176 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992ceab-24cf-421e-8e2e-37bab09c9337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:33.868870 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992ceab-24cf-421e-8e2e-37bab09c9337 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:33.869154 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992ceab-24cf-421e-8e2e-37bab09c9337 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:33.869428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992ceab-24cf-421e-8e2e-37bab09c9337 None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:33.870216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992ceab-24cf-421e-8e2e-37bab09c9337 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:33.870528 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992ceab-24cf-421e-8e2e-37bab09c9337 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:33.872000 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992ceab-24cf-421e-8e2e-37bab09c9337 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:33.873836 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e939b0-5ed0-470f-a860-a24ebc3b0fa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:33.875360 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e939b0-5ed0-470f-a860-a24ebc3b0fa6 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:33.890711 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 293/582] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:33.891350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992ceab-24cf-421e-8e2e-37bab09c9337 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:33.894605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-a6e939b0-5ed0-470f-a860-a24ebc3b0fa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:33.894605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:33.894605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:33.894605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:33.894605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:33.894605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:33.894605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:33.894605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:33.894605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:33.894605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:33.894605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:33.894605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:33.894605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:33.894605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:33.894605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:33.895662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:33.895662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:33.895662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:33.895662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:33.895662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:33.895662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:33.895662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:33.895662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:33.895662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:33.895662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:33.895662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:33.895662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:33.895662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:33.895662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:33.895662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:33.895662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:33.895662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:33.896677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:33.896677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:33.896677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:33.896677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:33.896677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 290/583] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:33.896965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e939b0-5ed0-470f-a860-a24ebc3b0fa6 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:33.934961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:33.966934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:34.007424 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992ceab-24cf-421e-8e2e-37bab09c9337 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:34.007673 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e939b0-5ed0-470f-a860-a24ebc3b0fa6 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:34.043994 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1cf886-13d9-4b75-a61c-0053a15ab918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:34.044829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cf886-13d9-4b75-a61c-0053a15ab918 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:34.045121 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cf886-13d9-4b75-a61c-0053a15ab918 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:34.045404 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cf886-13d9-4b75-a61c-0053a15ab918 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:34.045501 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c7be05-b243-4ac6-ab1e-4e318b05f320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:34.046074 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7be05-b243-4ac6-ab1e-4e318b05f320 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:34.046362 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7be05-b243-4ac6-ab1e-4e318b05f320 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:34.046639 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7be05-b243-4ac6-ab1e-4e318b05f320 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:34.047484 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c7be05-b243-4ac6-ab1e-4e318b05f320 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:34.047792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c7be05-b243-4ac6-ab1e-4e318b05f320 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:34.049271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c7be05-b243-4ac6-ab1e-4e318b05f320 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:34.057649 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1cf886-13d9-4b75-a61c-0053a15ab918 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:34.059199 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1cf886-13d9-4b75-a61c-0053a15ab918 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:34.069202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 294/584] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 23 22:27:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:34.070694 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c7be05-b243-4ac6-ab1e-4e318b05f320 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:34.083230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-ef1cf886-13d9-4b75-a61c-0053a15ab918 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:34.083230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:34.083230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:34.083230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:34.083230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:34.083230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:34.083230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:34.083230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:34.083230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:34.083230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:34.083230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:34.083230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:34.083230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:34.083230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:34.083230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:34.084119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:34.084119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:34.084119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:34.084119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:34.084119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:34.084119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:34.084119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:34.084119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:34.084119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:34.084119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:34.084119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:34.084119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:34.084119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:34.084119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:34.084119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:34.084119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:34.084119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:34.085690 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:34.085690 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:34.085690 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:34.085690 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:34.085690 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 291/585] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:34.087022 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1cf886-13d9-4b75-a61c-0053a15ab918 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:34.175488 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:34.184971 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c7be05-b243-4ac6-ab1e-4e318b05f320 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:34.195047 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1cf886-13d9-4b75-a61c-0053a15ab918 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:34.214642 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:34.220584 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf7a4c-a76c-4d09-b541-0ee0679967f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:34.221499 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf7a4c-a76c-4d09-b541-0ee0679967f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:34.221788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf7a4c-a76c-4d09-b541-0ee0679967f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:34.222053 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf7a4c-a76c-4d09-b541-0ee0679967f7 None admin] PATH_INFO: `/v3/projects/14265887fea245a5a37abbb3a116e352/users/159cc8befc3a4d57bbd313832084db62/roles/698ff821321a43d494fe8940cdd08564` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:34.232072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba75e8e-b64e-4566-9c7f-10be0b092c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:34.232841 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba75e8e-b64e-4566-9c7f-10be0b092c87 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:34.233129 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba75e8e-b64e-4566-9c7f-10be0b092c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:34.233406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba75e8e-b64e-4566-9c7f-10be0b092c87 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:34.234128 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba75e8e-b64e-4566-9c7f-10be0b092c87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:34.234439 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba75e8e-b64e-4566-9c7f-10be0b092c87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:34.235829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba75e8e-b64e-4566-9c7f-10be0b092c87 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:34.238252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf7a4c-a76c-4d09-b541-0ee0679967f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=14265887fea245a5a37abbb3a116e352, user_id=159cc8befc3a4d57bbd313832084db62, role_id=698ff821321a43d494fe8940cdd08564)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:34.239716 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf7a4c-a76c-4d09-b541-0ee0679967f7 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:34.248316 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 292/586] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:34.249281 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba75e8e-b64e-4566-9c7f-10be0b092c87 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:34.300414 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 295/587] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:27:34 2022] PUT /identity/v3/projects/14265887fea245a5a37abbb3a116e352/users/159cc8befc3a4d57bbd313832084db62/roles/698ff821321a43d494fe8940cdd08564 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:27:34.301637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf7a4c-a76c-4d09-b541-0ee0679967f7 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:34.379589 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba75e8e-b64e-4566-9c7f-10be0b092c87 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:34.461045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208bb8e2-c3e8-47d0-a0aa-4eed1795f848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:34.461045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bb8e2-c3e8-47d0-a0aa-4eed1795f848 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:34.461045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bb8e2-c3e8-47d0-a0aa-4eed1795f848 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:34.461441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bb8e2-c3e8-47d0-a0aa-4eed1795f848 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:34.468449 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf7a4c-a76c-4d09-b541-0ee0679967f7 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:34.473933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208bb8e2-c3e8-47d0-a0aa-4eed1795f848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:34.475403 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208bb8e2-c3e8-47d0-a0aa-4eed1795f848 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:34.497265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-208bb8e2-c3e8-47d0-a0aa-4eed1795f848 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:34.497265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:34.497265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:34.497265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:34.497265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:34.497265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:34.497265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:34.497265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:34.497265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:34.497265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:34.497265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:34.497265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:34.497265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:34.497265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:34.497265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:34.498412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:34.498412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:34.498412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:34.498412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:34.498412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:34.498412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:34.498412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:34.498412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:34.498412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:34.498412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:34.498412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:34.498412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:34.498412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:34.498412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:34.498412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:34.498412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:34.498412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:34.499501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:34.499501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:34.499501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:34.499501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:34.499501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 293/588] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:34.499842 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208bb8e2-c3e8-47d0-a0aa-4eed1795f848 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:34.518891 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f7659-3f89-4ca4-8487-e9eca73f9d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:34.519932 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f7659-3f89-4ca4-8487-e9eca73f9d36 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:34.520421 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f7659-3f89-4ca4-8487-e9eca73f9d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:34.520959 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f7659-3f89-4ca4-8487-e9eca73f9d36 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:34.521904 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f7659-3f89-4ca4-8487-e9eca73f9d36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:34.522416 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f7659-3f89-4ca4-8487-e9eca73f9d36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:34.524076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f7659-3f89-4ca4-8487-e9eca73f9d36 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:34.535648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:34.537403 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 296/589] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:34.543861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f7659-3f89-4ca4-8487-e9eca73f9d36 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:34.547160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:34.628539 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208bb8e2-c3e8-47d0-a0aa-4eed1795f848 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:34.659064 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f7659-3f89-4ca4-8487-e9eca73f9d36 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:34.669669 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0339dc-08c2-4680-9278-5ffeae1159f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:34.670750 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0339dc-08c2-4680-9278-5ffeae1159f4 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:34.671208 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0339dc-08c2-4680-9278-5ffeae1159f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:34.671663 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0339dc-08c2-4680-9278-5ffeae1159f4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:34.680515 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0339dc-08c2-4680-9278-5ffeae1159f4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:34.682244 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0339dc-08c2-4680-9278-5ffeae1159f4 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:34.697909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-bd0339dc-08c2-4680-9278-5ffeae1159f4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 22:27:34.697909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:34.697909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:34.697909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:34.697909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:34.697909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:34.697909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:34.697909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:34.697909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:34.697909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:34.697909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:34.697909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:34.697909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:34.697909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:34.697909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:34.697909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:34.698970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:34.698970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:34.698970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:34.698970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:34.698970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:34.698970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:34.698970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:34.698970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:34.698970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:34.698970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:34.698970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:34.698970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:34.698970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:34.698970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:34.698970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:34.698970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:34.698970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:34.698970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:34.699907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:34.699907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:34.699907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:34.699907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:34.699907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:34.699907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:34.699907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:34.699907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:34.699907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 22:27:34.699907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:34.702552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 294/590] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:34.702771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfd45e-9f0c-4205-803c-b71427bd4e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:34.703519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfd45e-9f0c-4205-803c-b71427bd4e4e None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:34.703811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfd45e-9f0c-4205-803c-b71427bd4e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:34.704088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfd45e-9f0c-4205-803c-b71427bd4e4e None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:34.704507 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0339dc-08c2-4680-9278-5ffeae1159f4 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:34.704853 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddfd45e-9f0c-4205-803c-b71427bd4e4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:34.705164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddfd45e-9f0c-4205-803c-b71427bd4e4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:34.707013 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddfd45e-9f0c-4205-803c-b71427bd4e4e None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:34.724667 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 297/591] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:34.725883 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfd45e-9f0c-4205-803c-b71427bd4e4e None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:34.799091 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:34.832617 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:34.846894 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0339dc-08c2-4680-9278-5ffeae1159f4 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:34.847837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfd45e-9f0c-4205-803c-b71427bd4e4e None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:34.885605 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee81f61-35ba-46a1-a812-c1765494ebd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:34.886411 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee81f61-35ba-46a1-a812-c1765494ebd0 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:34.886804 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee81f61-35ba-46a1-a812-c1765494ebd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:34.887185 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee81f61-35ba-46a1-a812-c1765494ebd0 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:34.888130 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee81f61-35ba-46a1-a812-c1765494ebd0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:34.888563 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee81f61-35ba-46a1-a812-c1765494ebd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:34.890153 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee81f61-35ba-46a1-a812-c1765494ebd0 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:34.894817 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54485082-c224-476c-a7d2-6c598c1bcb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:34.895793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54485082-c224-476c-a7d2-6c598c1bcb85 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:34.896248 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54485082-c224-476c-a7d2-6c598c1bcb85 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:34.896693 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54485082-c224-476c-a7d2-6c598c1bcb85 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:34.905970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54485082-c224-476c-a7d2-6c598c1bcb85 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:34.907891 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54485082-c224-476c-a7d2-6c598c1bcb85 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:34.909344 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 298/592] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 23 22:27:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:34.910299 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee81f61-35ba-46a1-a812-c1765494ebd0 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:34.926329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-54485082-c224-476c-a7d2-6c598c1bcb85 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 22:27:34.926329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:34.926329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:34.926329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:34.926329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:34.926329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:34.926329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:34.926329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:34.926329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:34.926329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:34.926329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:34.926329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:34.926329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:34.926329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:34.926329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:34.926329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:34.927496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:34.927496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:34.927496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:34.927496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:34.927496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:34.927496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:34.927496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:34.927496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:34.927496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:34.927496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:34.927496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:34.927496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:34.927496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:34.927496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:34.927496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:34.927496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:34.927496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:34.927496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:34.929207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:34.929207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:34.929207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:34.929207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:34.929207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:34.929207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:34.929207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:34.929207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:34.929207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 22:27:34.929207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:34.932921 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 295/593] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:27:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:34.933927 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54485082-c224-476c-a7d2-6c598c1bcb85 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:35.030133 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee81f61-35ba-46a1-a812-c1765494ebd0 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:35.051802 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:35.067273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9102a5-6485-4f81-8221-36c769617b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:35.068613 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9102a5-6485-4f81-8221-36c769617b5b None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:35.069006 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9102a5-6485-4f81-8221-36c769617b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:35.069311 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9102a5-6485-4f81-8221-36c769617b5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:35.077044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9102a5-6485-4f81-8221-36c769617b5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:35.078618 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9102a5-6485-4f81-8221-36c769617b5b None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:35.079625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54485082-c224-476c-a7d2-6c598c1bcb85 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:35.092818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-6f9102a5-6485-4f81-8221-36c769617b5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:35.092818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:35.092818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:35.092818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:35.092818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:35.092818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:35.092818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:35.092818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:35.092818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:35.092818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:35.092818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:35.092818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:35.092818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:35.092818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:35.092818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:35.093773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:35.093773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:35.093773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:35.093773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:35.093773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:35.093773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:35.093773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:35.093773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:35.093773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:35.093773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:35.093773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:35.093773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:35.093773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:35.093773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:35.093773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:35.093773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:35.093773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:35.094664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:35.094664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:35.094664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:35.094664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:35.094664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:35.094664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:35.094664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:35.094664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:35.099224 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 299/594] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:35.100030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9102a5-6485-4f81-8221-36c769617b5b None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:35.126459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4141cf-a53c-4eb4-97d6-859e7d8c7b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:35.127769 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4141cf-a53c-4eb4-97d6-859e7d8c7b5b None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:35.128229 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4141cf-a53c-4eb4-97d6-859e7d8c7b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:35.128678 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4141cf-a53c-4eb4-97d6-859e7d8c7b5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:35.144534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4141cf-a53c-4eb4-97d6-859e7d8c7b5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:35.146739 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4141cf-a53c-4eb4-97d6-859e7d8c7b5b None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:35.171516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-0a4141cf-a53c-4eb4-97d6-859e7d8c7b5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:35.171516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:35.171516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:35.171516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:35.171516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:35.171516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:35.171516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:35.171516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:35.171516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:35.171516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:35.171516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:35.171516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:35.171516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:35.171516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:35.171516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:35.172654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:35.172654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:35.172654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:35.172654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:35.172654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:35.172654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:35.172654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:35.172654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:35.172654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:35.172654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:35.172654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:35.172654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:35.172654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:35.172654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:35.172654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:35.172654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:35.172654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:35.173776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:35.173776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:35.173776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:35.173776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:35.175796 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 296/595] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:35.177406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4141cf-a53c-4eb4-97d6-859e7d8c7b5b None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:35.181634 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:35.216080 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9102a5-6485-4f81-8221-36c769617b5b None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:35.256461 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0c521-129b-437c-aa68-10fbf0be108e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:35.257891 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0c521-129b-437c-aa68-10fbf0be108e None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:35.258342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0c521-129b-437c-aa68-10fbf0be108e None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:35.258686 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0c521-129b-437c-aa68-10fbf0be108e None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:35.259588 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0c521-129b-437c-aa68-10fbf0be108e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:35.260023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0c521-129b-437c-aa68-10fbf0be108e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:35.261871 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0c521-129b-437c-aa68-10fbf0be108e None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:35.295639 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 300/596] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Jun 23 22:27:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:35.296781 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0c521-129b-437c-aa68-10fbf0be108e None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:35.318859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4141cf-a53c-4eb4-97d6-859e7d8c7b5b None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:35.341866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:35.365041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263d8f1-9d6f-4197-93d6-28a3f1e81367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:35.365951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263d8f1-9d6f-4197-93d6-28a3f1e81367 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:35.367054 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263d8f1-9d6f-4197-93d6-28a3f1e81367 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:35.367514 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263d8f1-9d6f-4197-93d6-28a3f1e81367 None admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:35.369038 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6263d8f1-9d6f-4197-93d6-28a3f1e81367 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:35.369504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6263d8f1-9d6f-4197-93d6-28a3f1e81367 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:35.371091 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6263d8f1-9d6f-4197-93d6-28a3f1e81367 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:35.387861 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 297/597] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:35.389577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263d8f1-9d6f-4197-93d6-28a3f1e81367 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:35.414798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0c521-129b-437c-aa68-10fbf0be108e None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:35.455130 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f66e8-c48b-43af-b489-c1695f0930e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:35.456328 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f66e8-c48b-43af-b489-c1695f0930e1 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:35.456761 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f66e8-c48b-43af-b489-c1695f0930e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:35.457153 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f66e8-c48b-43af-b489-c1695f0930e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:35.458300 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f66e8-c48b-43af-b489-c1695f0930e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:35.458719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f66e8-c48b-43af-b489-c1695f0930e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:35.460743 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f66e8-c48b-43af-b489-c1695f0930e1 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:35.472432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:35.477805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 301/598] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:35.482208 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f66e8-c48b-43af-b489-c1695f0930e1 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:35.513844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263d8f1-9d6f-4197-93d6-28a3f1e81367 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:35.552915 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7381f78-d2d2-42f3-bb04-d36afe071d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:35.554706 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7381f78-d2d2-42f3-bb04-d36afe071d32 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:35.555070 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7381f78-d2d2-42f3-bb04-d36afe071d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:35.555420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7381f78-d2d2-42f3-bb04-d36afe071d32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:35.567395 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7381f78-d2d2-42f3-bb04-d36afe071d32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:35.569140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7381f78-d2d2-42f3-bb04-d36afe071d32 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:35.587006 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:35.588304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-d7381f78-d2d2-42f3-bb04-d36afe071d32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:35.588304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:35.588304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:35.588304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:35.588304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:35.588304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:35.588304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:35.588304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:35.588304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:35.588304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:35.588304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:35.588304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:35.588304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:35.588304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:35.588304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:35.589336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:35.589336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:35.589336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:35.589336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:35.589336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:35.589336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:35.589336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:35.589336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:35.589336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:35.589336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:35.589336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:35.589336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:35.589336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:35.589336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:35.589336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:35.589336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:35.589336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:35.590394 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:35.590394 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:35.590394 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:35.590394 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:35.590394 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 298/599] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:35.590746 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7381f78-d2d2-42f3-bb04-d36afe071d32 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:35.594433 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f66e8-c48b-43af-b489-c1695f0930e1 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:35.631572 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3da06-9366-4777-8005-17bb339662aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:35.632976 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3da06-9366-4777-8005-17bb339662aa None admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:35.633411 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3da06-9366-4777-8005-17bb339662aa None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:35.633856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3da06-9366-4777-8005-17bb339662aa None admin] PATH_INFO: `/v3/projects/14265887fea245a5a37abbb3a116e352/users/5d1911bfac984e058b0b11bb6400e782/roles/698ff821321a43d494fe8940cdd08564` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:35.650789 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f3da06-9366-4777-8005-17bb339662aa None admin] RBAC: Authorizing `identity:create_grant(project_id=14265887fea245a5a37abbb3a116e352, user_id=5d1911bfac984e058b0b11bb6400e782, role_id=698ff821321a43d494fe8940cdd08564)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:35.652691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f3da06-9366-4777-8005-17bb339662aa None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:35.690968 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7381f78-d2d2-42f3-bb04-d36afe071d32 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:35.704221 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:35.720740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 302/600] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:27:35 2022] PUT /identity/v3/projects/14265887fea245a5a37abbb3a116e352/users/5d1911bfac984e058b0b11bb6400e782/roles/698ff821321a43d494fe8940cdd08564 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:27:35.720740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ad9e1-4822-4456-9628-3558bf4f4b3b None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:35.720740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ad9e1-4822-4456-9628-3558bf4f4b3b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:35.720740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ad9e1-4822-4456-9628-3558bf4f4b3b None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:35.720740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 303/601] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:35.720740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ad9e1-4822-4456-9628-3558bf4f4b3b None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:35.733703 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c5050-c230-4de0-aa03-df322105fae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:35.734591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c5050-c230-4de0-aa03-df322105fae4 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:35.734877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c5050-c230-4de0-aa03-df322105fae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:35.735200 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c5050-c230-4de0-aa03-df322105fae4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:35.743560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c5050-c230-4de0-aa03-df322105fae4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:35.745241 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c5050-c230-4de0-aa03-df322105fae4 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:35.762630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-ff3c5050-c230-4de0-aa03-df322105fae4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:35.762630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:35.762630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:35.762630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:35.762630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:35.762630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:35.762630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:35.762630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:35.762630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:35.762630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:35.762630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:35.762630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:35.762630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:35.762630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:35.762630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:35.763635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:35.763635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:35.763635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:35.763635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:35.763635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:35.763635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:35.763635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:35.763635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:35.763635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:35.763635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:35.763635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:35.763635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:35.763635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:35.763635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:35.763635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:35.763635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:35.763635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:35.764565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:35.764565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:35.764565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:35.764565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:35.764565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:35.764565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:35.764565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:35.764565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:35.764565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 299/602] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:35.765596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fa144-6db2-44fe-bd6f-552fe4593a5e None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:35.765596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fa144-6db2-44fe-bd6f-552fe4593a5e None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:35.765847 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fa144-6db2-44fe-bd6f-552fe4593a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:35.790242 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-bd7fa144-6db2-44fe-bd6f-552fe4593a5e None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:35.791542 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-bd7fa144-6db2-44fe-bd6f-552fe4593a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:35.870311 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ad9e1-4822-4456-9628-3558bf4f4b3b None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:35.874725 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 300/603] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:35.875596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fa144-6db2-44fe-bd6f-552fe4593a5e None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:35.908053 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea25fb2-b459-4a02-b5f8-5beb669c0928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:35.908853 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea25fb2-b459-4a02-b5f8-5beb669c0928 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:35.909214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea25fb2-b459-4a02-b5f8-5beb669c0928 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:35.909567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea25fb2-b459-4a02-b5f8-5beb669c0928 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:35.910414 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea25fb2-b459-4a02-b5f8-5beb669c0928 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:35.910795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea25fb2-b459-4a02-b5f8-5beb669c0928 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:35.912262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea25fb2-b459-4a02-b5f8-5beb669c0928 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:35.931005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 304/604] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:35.932261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea25fb2-b459-4a02-b5f8-5beb669c0928 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:35.944067 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:36.007471 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fa144-6db2-44fe-bd6f-552fe4593a5e None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:36.024972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:36.043304 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea25fb2-b459-4a02-b5f8-5beb669c0928 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:36.053836 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58159d8c-0697-4225-84d8-60be6ddcaded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:36.054528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58159d8c-0697-4225-84d8-60be6ddcaded None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:36.054796 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58159d8c-0697-4225-84d8-60be6ddcaded None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:36.055132 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58159d8c-0697-4225-84d8-60be6ddcaded None admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:36.056314 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58159d8c-0697-4225-84d8-60be6ddcaded None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:36.056827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58159d8c-0697-4225-84d8-60be6ddcaded None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:36.058425 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58159d8c-0697-4225-84d8-60be6ddcaded None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:36.074558 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 301/605] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:36.075644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58159d8c-0697-4225-84d8-60be6ddcaded None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:36.083455 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f6db0-af11-4875-8065-ae5da70fb229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:36.084793 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f6db0-af11-4875-8065-ae5da70fb229 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:36.085861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f6db0-af11-4875-8065-ae5da70fb229 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:36.086455 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f6db0-af11-4875-8065-ae5da70fb229 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:36.098389 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f6db0-af11-4875-8065-ae5da70fb229 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:36.106852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-1f9f6db0-af11-4875-8065-ae5da70fb229 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:36.106852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:36.106852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:36.106852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:36.106852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:36.106852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:36.106852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:36.106852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:36.106852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:36.106852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:36.106852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:36.106852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:36.106852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:36.106852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:36.106852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:36.106852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:36.107805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:36.107805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:36.107805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:36.107805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:36.107805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:36.107805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:36.107805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:36.107805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:36.107805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:36.107805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:36.107805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:36.107805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:36.107805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:36.107805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:36.107805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:36.107805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:36.108866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:36.108866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:36.108866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:36.108866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:36.108866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:36.108866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:36.108866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:36.108866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:36.108866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:36.108866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:36.108866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:36.108866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:36.108866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 305/606] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:27:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:36.109553 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f6db0-af11-4875-8065-ae5da70fb229 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:36.184601 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:36.216530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58159d8c-0697-4225-84d8-60be6ddcaded None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:36.219203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f6db0-af11-4875-8065-ae5da70fb229 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:36.262747 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f74ee8-dc36-4e9e-9baf-61903cfb6d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:36.263520 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7300d-849e-42ca-9feb-ceeadb28d837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:36.264208 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f74ee8-dc36-4e9e-9baf-61903cfb6d05 None admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:36.264335 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7300d-849e-42ca-9feb-ceeadb28d837 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:36.264606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7300d-849e-42ca-9feb-ceeadb28d837 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:36.264939 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7300d-849e-42ca-9feb-ceeadb28d837 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:36.265238 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f74ee8-dc36-4e9e-9baf-61903cfb6d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:36.265705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f74ee8-dc36-4e9e-9baf-61903cfb6d05 None admin] PATH_INFO: `/v3/projects/14265887fea245a5a37abbb3a116e352/users/983801bb18924fe4b3150693765eb13b/roles/698ff821321a43d494fe8940cdd08564` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:36.273569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a7300d-849e-42ca-9feb-ceeadb28d837 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:36.275580 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a7300d-849e-42ca-9feb-ceeadb28d837 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:36.282033 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f74ee8-dc36-4e9e-9baf-61903cfb6d05 None admin] RBAC: Authorizing `identity:create_grant(project_id=14265887fea245a5a37abbb3a116e352, user_id=983801bb18924fe4b3150693765eb13b, role_id=698ff821321a43d494fe8940cdd08564)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:36.283699 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f74ee8-dc36-4e9e-9baf-61903cfb6d05 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:36.293650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-10a7300d-849e-42ca-9feb-ceeadb28d837 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:36.293650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:36.293650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:36.293650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:36.293650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:36.293650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:36.293650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:36.293650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:36.293650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:36.293650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:36.293650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:36.293650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:36.293650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:36.293650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:36.293650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:36.330511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:36.330511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:36.330511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:36.330511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:36.330511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:36.330511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:36.330511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:36.330511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:36.330511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:36.330511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:36.330511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:36.330511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:36.330511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:36.330511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:36.330511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:36.330511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:36.330511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:36.331932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:36.331932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:36.331932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:36.331932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:36.331932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:36.331932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:36.331932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:36.331932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:36.331932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 302/607] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:36.331932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7300d-849e-42ca-9feb-ceeadb28d837 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:36.331932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:36.409856 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7300d-849e-42ca-9feb-ceeadb28d837 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:36.416278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 306/608] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:27:36 2022] PUT /identity/v3/projects/14265887fea245a5a37abbb3a116e352/users/983801bb18924fe4b3150693765eb13b/roles/698ff821321a43d494fe8940cdd08564 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:27:36.417582 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f74ee8-dc36-4e9e-9baf-61903cfb6d05 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:36.459616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b32202-8e31-4dfe-bcf4-24972d781bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:36.460775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b32202-8e31-4dfe-bcf4-24972d781bfe None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:36.461289 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b32202-8e31-4dfe-bcf4-24972d781bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:36.461793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b32202-8e31-4dfe-bcf4-24972d781bfe None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:36.463422 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b32202-8e31-4dfe-bcf4-24972d781bfe None admin] RBAC: Adding query filter params (name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:36.464157 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b32202-8e31-4dfe-bcf4-24972d781bfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:36.466149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b32202-8e31-4dfe-bcf4-24972d781bfe None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:36.478167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 303/609] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:27:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:36.483929 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b32202-8e31-4dfe-bcf4-24972d781bfe None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:36.589468 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f74ee8-dc36-4e9e-9baf-61903cfb6d05 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:36.613482 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:36.616368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:36.631032 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b32202-8e31-4dfe-bcf4-24972d781bfe None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:36.635976 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8bd88-e6fb-4abe-9381-7d7231b2b7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:36.636680 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8bd88-e6fb-4abe-9381-7d7231b2b7df None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:36.636969 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8bd88-e6fb-4abe-9381-7d7231b2b7df None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:36.637248 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8bd88-e6fb-4abe-9381-7d7231b2b7df None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:36.638049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8bd88-e6fb-4abe-9381-7d7231b2b7df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:36.638359 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8bd88-e6fb-4abe-9381-7d7231b2b7df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:36.639789 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8bd88-e6fb-4abe-9381-7d7231b2b7df None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:36.660392 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 307/610] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:36.660392 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c227a-d2ed-4203-997f-1937e2083a30 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:36.660392 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c227a-d2ed-4203-997f-1937e2083a30 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:36.660392 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c227a-d2ed-4203-997f-1937e2083a30 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:36.660793 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 308/611] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:36.669184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c227a-d2ed-4203-997f-1937e2083a30 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:36.687790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e7571a-0601-4700-bd32-c7a467cca5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:36.689214 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7571a-0601-4700-bd32-c7a467cca5cc None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:36.689914 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7571a-0601-4700-bd32-c7a467cca5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:36.690339 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7571a-0601-4700-bd32-c7a467cca5cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:36.707081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e7571a-0601-4700-bd32-c7a467cca5cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:36.709416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e7571a-0601-4700-bd32-c7a467cca5cc None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:36.733726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-36e7571a-0601-4700-bd32-c7a467cca5cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:36.733726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:36.733726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:36.733726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:36.733726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:36.733726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:36.733726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:36.733726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:36.733726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:36.733726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:36.733726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:36.733726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:36.733726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:36.733726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:36.733726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:36.734845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:36.734845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:36.734845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:36.734845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:36.734845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:36.734845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:36.734845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:36.734845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:36.734845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:36.734845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:36.734845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:36.734845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:36.734845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:36.734845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:36.734845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:36.734845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:36.734845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:36.735953 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:36.735953 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:36.735953 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:36.735953 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:36.738336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 304/612] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:36.740385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbfa2d-7303-47d2-a91f-e7ab8d994a7f None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:36.740877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbfa2d-7303-47d2-a91f-e7ab8d994a7f None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:36.741528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbfa2d-7303-47d2-a91f-e7ab8d994a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:36.765320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-7afbfa2d-7303-47d2-a91f-e7ab8d994a7f None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:36.767098 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-7afbfa2d-7303-47d2-a91f-e7ab8d994a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:36.801158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c227a-d2ed-4203-997f-1937e2083a30 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:36.848474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 305/613] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:36.848855 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57861a2-54e7-4f9f-8e90-53d7b974629e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:36.849991 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57861a2-54e7-4f9f-8e90-53d7b974629e None admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:36.850413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57861a2-54e7-4f9f-8e90-53d7b974629e None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:36.851484 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57861a2-54e7-4f9f-8e90-53d7b974629e None admin] PATH_INFO: `/v3/projects/14265887fea245a5a37abbb3a116e352/users/ec7412a9a3dc46369fe7b4771e3437de/roles/698ff821321a43d494fe8940cdd08564` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:36.855740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbfa2d-7303-47d2-a91f-e7ab8d994a7f None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:36.870893 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57861a2-54e7-4f9f-8e90-53d7b974629e None admin] RBAC: Authorizing `identity:create_grant(project_id=14265887fea245a5a37abbb3a116e352, user_id=ec7412a9a3dc46369fe7b4771e3437de, role_id=698ff821321a43d494fe8940cdd08564)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:36.872568 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57861a2-54e7-4f9f-8e90-53d7b974629e None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:36.938460 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 309/614] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:27:36 2022] PUT /identity/v3/projects/14265887fea245a5a37abbb3a116e352/users/ec7412a9a3dc46369fe7b4771e3437de/roles/698ff821321a43d494fe8940cdd08564 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:27:36.944962 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57861a2-54e7-4f9f-8e90-53d7b974629e None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:36.947628 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:36.990046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbfa2d-7303-47d2-a91f-e7ab8d994a7f None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:37.006122 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:37.039375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55efa79-c5a8-477b-8988-95751cc0810b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:37.042381 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55efa79-c5a8-477b-8988-95751cc0810b None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:37.045465 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55efa79-c5a8-477b-8988-95751cc0810b None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:37.045967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55efa79-c5a8-477b-8988-95751cc0810b None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:37.046905 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55efa79-c5a8-477b-8988-95751cc0810b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:37.048002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55efa79-c5a8-477b-8988-95751cc0810b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:37.051146 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55efa79-c5a8-477b-8988-95751cc0810b None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:37.074671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 306/615] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:37.080346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55efa79-c5a8-477b-8988-95751cc0810b None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:37.161269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57861a2-54e7-4f9f-8e90-53d7b974629e None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:37.221897 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7968be-4d41-4d79-af7c-1a8f612908f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:37.222871 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7968be-4d41-4d79-af7c-1a8f612908f3 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:37.223263 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7968be-4d41-4d79-af7c-1a8f612908f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:37.223666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7968be-4d41-4d79-af7c-1a8f612908f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:37.235631 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7968be-4d41-4d79-af7c-1a8f612908f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:37.240707 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55efa79-c5a8-477b-8988-95751cc0810b None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:37.244065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-0b7968be-4d41-4d79-af7c-1a8f612908f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:37.244065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:37.244065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:37.244065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:37.244065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:37.244065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:37.244065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:37.244065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:37.244065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:37.244065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:37.244065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:37.244065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:37.244065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:37.244065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:37.244065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:37.244065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:37.245193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:37.245193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:37.245193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:37.245193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:37.245193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:37.245193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:37.245193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:37.245193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:37.245193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:37.245193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:37.245193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:37.245193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:37.245193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:37.245193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:37.245193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:37.245193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:37.246229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:37.246229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:37.246229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:37.246229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:37.246229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:37.246229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:37.246229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:37.246229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:37.246229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:37.246229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:37.246229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:37.246229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:37.253705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 310/616] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:27:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:37.254742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7968be-4d41-4d79-af7c-1a8f612908f3 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:37.276145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313c735-1340-4c49-9b1c-59fce9b7fd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:37.276984 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313c735-1340-4c49-9b1c-59fce9b7fd94 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:37.277274 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313c735-1340-4c49-9b1c-59fce9b7fd94 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:37.277552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313c735-1340-4c49-9b1c-59fce9b7fd94 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:37.279636 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:37.286915 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313c735-1340-4c49-9b1c-59fce9b7fd94 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:37.290500 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313c735-1340-4c49-9b1c-59fce9b7fd94 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:37.309619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-3313c735-1340-4c49-9b1c-59fce9b7fd94 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:27:37.309619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:37.309619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:37.309619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:37.309619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:37.309619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:37.309619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:37.309619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:37.309619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:37.309619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:37.309619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:37.309619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:37.309619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:37.309619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:37.309619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:37.309619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:37.310784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:37.310784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:37.310784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:37.310784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:37.310784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:37.310784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:37.310784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:37.310784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:37.310784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:37.310784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:37.310784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:37.310784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:37.310784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:37.310784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:37.310784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:37.310784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:37.310784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:37.310784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:37.311934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:37.311934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:37.311934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:37.311934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:37.311934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:37.311934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:37.311934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:37.311934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:37.311934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:27:37.311934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:37.316307 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 307/617] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:27:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:37.322647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313c735-1340-4c49-9b1c-59fce9b7fd94 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:37.400557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:37.442813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7968be-4d41-4d79-af7c-1a8f612908f3 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:37.444427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313c735-1340-4c49-9b1c-59fce9b7fd94 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:37.483729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847e7c1-a5c0-46ad-be95-ce17bf479946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:37.484639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847e7c1-a5c0-46ad-be95-ce17bf479946 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:37.485045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847e7c1-a5c0-46ad-be95-ce17bf479946 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:37.485576 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847e7c1-a5c0-46ad-be95-ce17bf479946 None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:37.486545 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847e7c1-a5c0-46ad-be95-ce17bf479946 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:37.486984 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847e7c1-a5c0-46ad-be95-ce17bf479946 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:37.488601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847e7c1-a5c0-46ad-be95-ce17bf479946 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:37.514923 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 308/618] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 23 22:27:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:37.520445 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464a820-05ea-4d62-a004-206303223336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:37.520898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847e7c1-a5c0-46ad-be95-ce17bf479946 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:37.521150 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464a820-05ea-4d62-a004-206303223336 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:37.521439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464a820-05ea-4d62-a004-206303223336 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:37.521707 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464a820-05ea-4d62-a004-206303223336 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:37.523026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464a820-05ea-4d62-a004-206303223336 None admin] RBAC: Adding query filter params (name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:37.523338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464a820-05ea-4d62-a004-206303223336 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:37.524769 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464a820-05ea-4d62-a004-206303223336 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:37.533966 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 311/619] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:27:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:37.543898 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464a820-05ea-4d62-a004-206303223336 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:37.580673 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:37.666052 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464a820-05ea-4d62-a004-206303223336 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:37.697780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847e7c1-a5c0-46ad-be95-ce17bf479946 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:37.740817 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2376843-0ec7-4f17-8ff6-f62c5aa6a468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:37.742190 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2376843-0ec7-4f17-8ff6-f62c5aa6a468 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:37.742483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2376843-0ec7-4f17-8ff6-f62c5aa6a468 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:37.742757 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2376843-0ec7-4f17-8ff6-f62c5aa6a468 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:37.762128 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315e1f7-3187-43b3-9050-7b7587e73d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:37.762948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315e1f7-3187-43b3-9050-7b7587e73d05 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:37.763233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315e1f7-3187-43b3-9050-7b7587e73d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:37.763509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315e1f7-3187-43b3-9050-7b7587e73d05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:37.770106 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2376843-0ec7-4f17-8ff6-f62c5aa6a468 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:37.771568 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2376843-0ec7-4f17-8ff6-f62c5aa6a468 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:37.789027 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315e1f7-3187-43b3-9050-7b7587e73d05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:37.790508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315e1f7-3187-43b3-9050-7b7587e73d05 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:37.800900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-e2376843-0ec7-4f17-8ff6-f62c5aa6a468 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:37.800900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:37.800900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:37.800900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:37.800900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:37.800900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:37.800900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:37.800900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:37.800900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:37.800900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:37.800900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:37.800900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:37.800900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:37.800900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:37.800900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:37.801783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:37.801783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:37.801783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:37.801783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:37.801783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:37.801783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:37.801783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:37.801783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:37.801783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:37.801783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:37.801783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:37.801783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:37.801783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:37.801783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:37.801783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:37.801783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:37.801783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:37.802668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:37.802668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:37.802668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:37.802668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:37.802892 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 312/620] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:37.808860 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2376843-0ec7-4f17-8ff6-f62c5aa6a468 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:37.811839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:37.829784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-a315e1f7-3187-43b3-9050-7b7587e73d05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:37.829784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:37.829784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:37.829784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:37.829784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:37.829784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:37.829784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:37.829784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:37.829784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:37.829784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:37.829784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:37.829784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:37.829784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:37.829784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:37.829784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:37.831875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:37.831875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:37.831875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:37.831875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:37.831875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:37.831875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:37.831875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:37.831875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:37.831875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:37.831875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:37.831875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:37.831875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:37.831875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:37.831875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:37.831875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:37.831875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:37.831875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:37.834180 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:37.834180 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:37.834180 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:37.834180 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:37.841038 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 309/621] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:37.846449 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315e1f7-3187-43b3-9050-7b7587e73d05 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:37.921884 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2376843-0ec7-4f17-8ff6-f62c5aa6a468 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:37.958844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22cfda-49de-419b-bd22-1671bb2fed84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:37.959442 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22cfda-49de-419b-bd22-1671bb2fed84 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:37.960259 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22cfda-49de-419b-bd22-1671bb2fed84 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:37.960594 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22cfda-49de-419b-bd22-1671bb2fed84 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:37.961404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22cfda-49de-419b-bd22-1671bb2fed84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:37.961719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22cfda-49de-419b-bd22-1671bb2fed84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:37.963158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22cfda-49de-419b-bd22-1671bb2fed84 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:37.977200 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 313/622] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:37.982496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22cfda-49de-419b-bd22-1671bb2fed84 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:38.032887 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:38.036461 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315e1f7-3187-43b3-9050-7b7587e73d05 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:38.055675 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:38.073015 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e52ab1-6c9f-4c1a-a829-23d50d1c1b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:38.073876 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e52ab1-6c9f-4c1a-a829-23d50d1c1b85 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:38.074014 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e52ab1-6c9f-4c1a-a829-23d50d1c1b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:38.074293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e52ab1-6c9f-4c1a-a829-23d50d1c1b85 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:38.075403 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e52ab1-6c9f-4c1a-a829-23d50d1c1b85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:38.076098 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e52ab1-6c9f-4c1a-a829-23d50d1c1b85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:38.077998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e52ab1-6c9f-4c1a-a829-23d50d1c1b85 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:38.090859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 310/623] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:27:38.092302 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b229b7-5647-4b1f-9228-4eeae0b34f3a None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:38.092700 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b229b7-5647-4b1f-9228-4eeae0b34f3a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:38.093064 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b229b7-5647-4b1f-9228-4eeae0b34f3a None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:38.094624 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 311/624] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:38.095899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b229b7-5647-4b1f-9228-4eeae0b34f3a None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:38.100115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22cfda-49de-419b-bd22-1671bb2fed84 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:38.142132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531ec7f-ee83-4877-af76-2064014285f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:38.143044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531ec7f-ee83-4877-af76-2064014285f8 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:38.143339 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531ec7f-ee83-4877-af76-2064014285f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:38.143624 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531ec7f-ee83-4877-af76-2064014285f8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:38.152652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531ec7f-ee83-4877-af76-2064014285f8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:38.155249 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531ec7f-ee83-4877-af76-2064014285f8 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:38.173651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-d531ec7f-ee83-4877-af76-2064014285f8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:27:38.173651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:38.173651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:38.173651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:38.173651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:38.173651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:38.173651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:38.173651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:38.173651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:38.173651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:38.173651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:38.173651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:38.173651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:38.173651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:38.173651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:38.173651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:38.174865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:38.174865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:38.174865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:38.174865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:38.174865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:38.174865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:38.174865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:38.174865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:38.174865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:38.174865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:38.174865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:38.174865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:38.174865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:38.174865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:38.174865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:38.174865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:38.174865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:38.174865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:38.176005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:38.176005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:38.176005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:38.176005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:38.176005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:38.176005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:38.176005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:38.176005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:38.176005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:27:38.176005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:38.179763 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 314/625] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 23 22:27:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:27:38.181178 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce62dfd-c773-4e35-a54d-911cb8071d40 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:38.181551 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce62dfd-c773-4e35-a54d-911cb8071d40 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:38.181840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce62dfd-c773-4e35-a54d-911cb8071d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:38.199658 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-0ce62dfd-c773-4e35-a54d-911cb8071d40 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:38.200998 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-0ce62dfd-c773-4e35-a54d-911cb8071d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:38.208297 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b229b7-5647-4b1f-9228-4eeae0b34f3a None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:38.245356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2e794-03cb-48c9-b361-04ebb8a188d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:38.246341 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2e794-03cb-48c9-b361-04ebb8a188d6 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:38.246740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2e794-03cb-48c9-b361-04ebb8a188d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:38.247136 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2e794-03cb-48c9-b361-04ebb8a188d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:38.254680 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2e794-03cb-48c9-b361-04ebb8a188d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:38.256359 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2e794-03cb-48c9-b361-04ebb8a188d6 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:38.273100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-65f2e794-03cb-48c9-b361-04ebb8a188d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:38.273100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:38.273100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:38.273100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:38.273100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:38.273100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:38.273100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:38.273100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:38.273100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:38.273100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:38.273100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:38.273100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:38.273100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:38.273100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:38.273100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:38.274279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:38.274279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:38.274279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:38.274279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:38.274279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:38.274279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:38.274279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:38.274279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:38.274279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:38.274279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:38.274279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:38.274279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:38.274279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:38.274279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:38.274279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:38.274279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:38.274279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:38.275341 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:38.275341 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:38.275341 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:38.275341 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:38.275341 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:38.275341 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:38.275341 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:38.275341 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:38.280620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 312/626] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:38.280620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2e794-03cb-48c9-b361-04ebb8a188d6 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:38.280868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 315/627] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:38.282713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce62dfd-c773-4e35-a54d-911cb8071d40 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:38.300079 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:38.344378 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:38.398661 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce62dfd-c773-4e35-a54d-911cb8071d40 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:38.405531 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2e794-03cb-48c9-b361-04ebb8a188d6 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:38.440869 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47385a3e-a033-45a7-8499-f5acd4c2d690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:38.441783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47385a3e-a033-45a7-8499-f5acd4c2d690 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:38.442334 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47385a3e-a033-45a7-8499-f5acd4c2d690 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:38.442868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47385a3e-a033-45a7-8499-f5acd4c2d690 None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:38.443926 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e912e-b44d-4150-814b-f8f3dec13cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:38.444060 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47385a3e-a033-45a7-8499-f5acd4c2d690 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:38.445101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47385a3e-a033-45a7-8499-f5acd4c2d690 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:38.445913 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e912e-b44d-4150-814b-f8f3dec13cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:38.446356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e912e-b44d-4150-814b-f8f3dec13cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:38.446660 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e912e-b44d-4150-814b-f8f3dec13cc0 None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:38.447620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9e912e-b44d-4150-814b-f8f3dec13cc0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:38.447848 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47385a3e-a033-45a7-8499-f5acd4c2d690 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:38.448143 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9e912e-b44d-4150-814b-f8f3dec13cc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:38.449980 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9e912e-b44d-4150-814b-f8f3dec13cc0 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:38.464536 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 316/628] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:38.465800 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47385a3e-a033-45a7-8499-f5acd4c2d690 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:38.480318 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 313/629] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 23 22:27:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:38.481549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e912e-b44d-4150-814b-f8f3dec13cc0 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:38.577695 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:38.602443 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47385a3e-a033-45a7-8499-f5acd4c2d690 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:38.609169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e912e-b44d-4150-814b-f8f3dec13cc0 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:38.621279 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:38.649846 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4697da-59a6-4cc7-9d9d-fb0d0fd41969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:38.650814 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4697da-59a6-4cc7-9d9d-fb0d0fd41969 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:38.651321 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4697da-59a6-4cc7-9d9d-fb0d0fd41969 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:38.651838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4697da-59a6-4cc7-9d9d-fb0d0fd41969 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:38.653245 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90081093-0941-4352-b373-77bd080ffbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:38.654464 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90081093-0941-4352-b373-77bd080ffbae None admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:38.654889 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90081093-0941-4352-b373-77bd080ffbae None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:38.655594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90081093-0941-4352-b373-77bd080ffbae None admin] PATH_INFO: `/v3/projects/14265887fea245a5a37abbb3a116e352/users/b321586cdb3d4d25904472aaa4adba71/roles/698ff821321a43d494fe8940cdd08564` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:38.665009 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4697da-59a6-4cc7-9d9d-fb0d0fd41969 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:38.672862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90081093-0941-4352-b373-77bd080ffbae None admin] RBAC: Authorizing `identity:create_grant(project_id=14265887fea245a5a37abbb3a116e352, user_id=b321586cdb3d4d25904472aaa4adba71, role_id=698ff821321a43d494fe8940cdd08564)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:38.674470 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90081093-0941-4352-b373-77bd080ffbae None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:38.674956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-9e4697da-59a6-4cc7-9d9d-fb0d0fd41969 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:38.674956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:38.674956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:38.674956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:38.674956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:38.674956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:38.674956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:38.674956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:38.674956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:38.674956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:38.674956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:38.674956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:38.674956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:38.674956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:38.674956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:38.674956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:38.676308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:38.676308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:38.676308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:38.676308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:38.676308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:38.676308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:38.676308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:38.676308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:38.676308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:38.676308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:38.676308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:38.676308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:38.676308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:38.676308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:38.676308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:38.676308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:38.677320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:38.677320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:38.677320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:38.677320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:38.677320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:38.677320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:38.677320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:38.677320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:38.677320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:38.677320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:38.677320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:38.677320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:38.683650 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 317/630] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:27:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:38.683650 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4697da-59a6-4cc7-9d9d-fb0d0fd41969 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:38.746897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 314/631] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:27:38 2022] PUT /identity/v3/projects/14265887fea245a5a37abbb3a116e352/users/b321586cdb3d4d25904472aaa4adba71/roles/698ff821321a43d494fe8940cdd08564 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:27:38.748275 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90081093-0941-4352-b373-77bd080ffbae None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:38.792563 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4697da-59a6-4cc7-9d9d-fb0d0fd41969 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:38.831661 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c8aa7-7741-416b-b7bc-7b5073c5b5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:38.832416 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c8aa7-7741-416b-b7bc-7b5073c5b5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:38.832825 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c8aa7-7741-416b-b7bc-7b5073c5b5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:38.833368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c8aa7-7741-416b-b7bc-7b5073c5b5f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:38.891496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90081093-0941-4352-b373-77bd080ffbae None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:38.892435 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:38.930084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:38.930276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c8aa7-7741-416b-b7bc-7b5073c5b5f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:38.931859 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c8aa7-7741-416b-b7bc-7b5073c5b5f2 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:38.939423 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3aee68-16ef-4435-a6ff-790cfe105e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:38.940089 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3aee68-16ef-4435-a6ff-790cfe105e36 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:38.940374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3aee68-16ef-4435-a6ff-790cfe105e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:38.941315 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3aee68-16ef-4435-a6ff-790cfe105e36 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:38.941498 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3aee68-16ef-4435-a6ff-790cfe105e36 None admin] RBAC: Adding query filter params (name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:38.941779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3aee68-16ef-4435-a6ff-790cfe105e36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:38.943166 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3aee68-16ef-4435-a6ff-790cfe105e36 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:38.960791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 315/632] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:27:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:38.963211 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdea7e-4575-4016-b110-90ae7aacf76c None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:38.967881 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdea7e-4575-4016-b110-90ae7aacf76c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:38.967881 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdea7e-4575-4016-b110-90ae7aacf76c None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:38.968659 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 316/633] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:38.973569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdea7e-4575-4016-b110-90ae7aacf76c None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:39.048925 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 318/634] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:39.050412 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21cb9a-1aa1-4dd7-a8a6-935435f82769 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:39.050647 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21cb9a-1aa1-4dd7-a8a6-935435f82769 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:39.050931 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21cb9a-1aa1-4dd7-a8a6-935435f82769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:39.068855 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-6a21cb9a-1aa1-4dd7-a8a6-935435f82769 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:39.070889 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6a21cb9a-1aa1-4dd7-a8a6-935435f82769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:39.092390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdea7e-4575-4016-b110-90ae7aacf76c None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:39.130255 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb109d2-b4e4-424b-8ff7-1f3e53e2e283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:39.131436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb109d2-b4e4-424b-8ff7-1f3e53e2e283 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:39.133761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb109d2-b4e4-424b-8ff7-1f3e53e2e283 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:39.134452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb109d2-b4e4-424b-8ff7-1f3e53e2e283 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:39.145715 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb109d2-b4e4-424b-8ff7-1f3e53e2e283 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:39.148112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb109d2-b4e4-424b-8ff7-1f3e53e2e283 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:39.149067 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 319/635] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:39.149971 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21cb9a-1aa1-4dd7-a8a6-935435f82769 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:39.152420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:39.170447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-2fb109d2-b4e4-424b-8ff7-1f3e53e2e283 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:39.170447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:39.170447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:39.170447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:39.170447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:39.170447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:39.170447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:39.170447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:39.170447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:39.170447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:39.170447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:39.170447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:39.170447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:39.170447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:39.170447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:39.171356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:39.171356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:39.171356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:39.171356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:39.171356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:39.171356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:39.171356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:39.171356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:39.171356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:39.171356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:39.171356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:39.171356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:39.171356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:39.171356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:39.171356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:39.171356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:39.171356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:39.172233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:39.172233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:39.172233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:39.172233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:39.175368 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 317/636] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:39.181668 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb109d2-b4e4-424b-8ff7-1f3e53e2e283 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:39.227718 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:39.295751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21cb9a-1aa1-4dd7-a8a6-935435f82769 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:39.305518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb109d2-b4e4-424b-8ff7-1f3e53e2e283 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:39.333006 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5133e-41b3-42ab-bec8-8b996b2f414e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:39.334244 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5133e-41b3-42ab-bec8-8b996b2f414e None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:39.334807 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5133e-41b3-42ab-bec8-8b996b2f414e None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:39.335371 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5133e-41b3-42ab-bec8-8b996b2f414e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:39.346623 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5133e-41b3-42ab-bec8-8b996b2f414e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:39.348785 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5133e-41b3-42ab-bec8-8b996b2f414e None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:39.349446 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac6c64-fa50-4c32-baff-404ca5fd4663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:39.350235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac6c64-fa50-4c32-baff-404ca5fd4663 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:39.351116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac6c64-fa50-4c32-baff-404ca5fd4663 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:39.351466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac6c64-fa50-4c32-baff-404ca5fd4663 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:39.352754 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac6c64-fa50-4c32-baff-404ca5fd4663 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:39.353025 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac6c64-fa50-4c32-baff-404ca5fd4663 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:39.354996 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac6c64-fa50-4c32-baff-404ca5fd4663 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:39.374746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-7aa5133e-41b3-42ab-bec8-8b996b2f414e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:39.374746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:39.374746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:39.374746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:39.374746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:39.374746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:39.374746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:39.374746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:39.374746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:39.374746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:39.374746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:39.374746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:39.374746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:39.374746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:39.374746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:39.375717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:39.375717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:39.375717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:39.375717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:39.375717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:39.375717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:39.375717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:39.375717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:39.375717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:39.375717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:39.375717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:39.375717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:39.375717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:39.375717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:39.375717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:39.375717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:39.375717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:39.376638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:39.376638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:39.376638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:39.376638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:39.376870 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 318/637] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:39.376870 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac6c64-fa50-4c32-baff-404ca5fd4663 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:39.377010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 320/638] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:39.379553 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5133e-41b3-42ab-bec8-8b996b2f414e None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:39.444851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:39.488132 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:39.491551 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac6c64-fa50-4c32-baff-404ca5fd4663 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:39.494103 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5133e-41b3-42ab-bec8-8b996b2f414e None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:39.526275 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d98e39-0654-4ab9-b973-796cea90f4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:39.527393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d98e39-0654-4ab9-b973-796cea90f4cb None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:39.527871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d98e39-0654-4ab9-b973-796cea90f4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:39.528216 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d98e39-0654-4ab9-b973-796cea90f4cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:39.532559 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165736c8-507f-4bb2-91b8-091b7e0d0d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:39.533437 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165736c8-507f-4bb2-91b8-091b7e0d0d8d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:39.533663 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165736c8-507f-4bb2-91b8-091b7e0d0d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:39.533942 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165736c8-507f-4bb2-91b8-091b7e0d0d8d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:39.540448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d98e39-0654-4ab9-b973-796cea90f4cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:39.542741 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165736c8-507f-4bb2-91b8-091b7e0d0d8d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:39.544187 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165736c8-507f-4bb2-91b8-091b7e0d0d8d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:39.548942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-65d98e39-0654-4ab9-b973-796cea90f4cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:39.548942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:39.548942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:39.548942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:39.548942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:39.548942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:39.548942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:39.548942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:39.548942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:39.548942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:39.548942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:39.548942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:39.548942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:39.548942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:39.548942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:39.548942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:39.550017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:39.550017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:39.550017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:39.550017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:39.550017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:39.550017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:39.550017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:39.550017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:39.550017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:39.550017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:39.550017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:39.550017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:39.550017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:39.550017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:39.550017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:39.550017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:39.550956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:39.550956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:39.550956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:39.550956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:39.550956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:39.550956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:39.550956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:39.550956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:39.550956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:39.550956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:39.550956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:39.550956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:39.553315 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 319/639] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:27:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:39.554255 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d98e39-0654-4ab9-b973-796cea90f4cb None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:39.559082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-165736c8-507f-4bb2-91b8-091b7e0d0d8d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 22:27:39.559082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:39.559082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:39.559082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:39.559082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:39.559082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:39.559082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:39.559082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:39.559082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:39.559082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:39.559082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:39.559082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:39.559082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:39.559082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:39.559082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:39.559082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:39.560216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:39.560216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:39.560216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:39.560216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:39.560216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:39.560216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:39.560216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:39.560216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:39.560216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:39.560216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:39.560216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:39.560216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:39.560216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:39.560216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:39.560216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:39.560216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:39.560216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:39.560216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:39.561418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:39.561418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:39.561418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:39.561418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:39.561418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:39.561418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:39.561418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:39.561418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:39.561418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 22:27:39.561418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:39.564703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 321/640] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:39.565994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b0c62-28be-4f3d-ad36-39bdda45df18 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:39.566288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b0c62-28be-4f3d-ad36-39bdda45df18 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:39.566566 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b0c62-28be-4f3d-ad36-39bdda45df18 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:39.567842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 322/641] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:39.568529 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b0c62-28be-4f3d-ad36-39bdda45df18 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:39.668730 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d98e39-0654-4ab9-b973-796cea90f4cb None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:39.680945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b0c62-28be-4f3d-ad36-39bdda45df18 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:39.695664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:39.703903 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9adea-b5f3-43ee-9076-fb5d1030cb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:39.704559 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9adea-b5f3-43ee-9076-fb5d1030cb01 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:39.704874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9adea-b5f3-43ee-9076-fb5d1030cb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:39.705153 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9adea-b5f3-43ee-9076-fb5d1030cb01 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:39.708392 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d9adea-b5f3-43ee-9076-fb5d1030cb01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:39.708392 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d9adea-b5f3-43ee-9076-fb5d1030cb01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:39.708392 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d9adea-b5f3-43ee-9076-fb5d1030cb01 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:39.716921 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36827c1e-6d22-4a08-b22a-041a6bb28738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:39.717579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36827c1e-6d22-4a08-b22a-041a6bb28738 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:39.717863 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36827c1e-6d22-4a08-b22a-041a6bb28738 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:39.718147 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36827c1e-6d22-4a08-b22a-041a6bb28738 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:39.718882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36827c1e-6d22-4a08-b22a-041a6bb28738 None admin] RBAC: Adding query filter params (name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:39.719187 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36827c1e-6d22-4a08-b22a-041a6bb28738 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:39.724808 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 320/642] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:39.725050 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36827c1e-6d22-4a08-b22a-041a6bb28738 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:39.725145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9adea-b5f3-43ee-9076-fb5d1030cb01 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:39.738855 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 323/643] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:27:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:39.738855 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb72ef-fb8f-42fd-a5da-634b2ad0e0ed None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:39.738855 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb72ef-fb8f-42fd-a5da-634b2ad0e0ed None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:39.738855 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb72ef-fb8f-42fd-a5da-634b2ad0e0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:39.751237 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:39.753748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-88cb72ef-fb8f-42fd-a5da-634b2ad0e0ed None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:39.754939 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-88cb72ef-fb8f-42fd-a5da-634b2ad0e0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:39.828739 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 324/644] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:39.828739 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb72ef-fb8f-42fd-a5da-634b2ad0e0ed None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:39.840780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9adea-b5f3-43ee-9076-fb5d1030cb01 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:39.876366 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e1a1a-ca74-4d3d-8f95-febe9ec746ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:39.877115 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e1a1a-ca74-4d3d-8f95-febe9ec746ac None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:39.877404 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e1a1a-ca74-4d3d-8f95-febe9ec746ac None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:39.877683 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e1a1a-ca74-4d3d-8f95-febe9ec746ac None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:39.878621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e1a1a-ca74-4d3d-8f95-febe9ec746ac None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:39.878937 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e1a1a-ca74-4d3d-8f95-febe9ec746ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:39.880354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e1a1a-ca74-4d3d-8f95-febe9ec746ac None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:39.899422 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 321/645] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 23 22:27:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:39.900312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e1a1a-ca74-4d3d-8f95-febe9ec746ac None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:39.947357 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb72ef-fb8f-42fd-a5da-634b2ad0e0ed None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:39.955119 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:39.984134 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba153ee-5c22-4252-a932-535e97e7b3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:39.985072 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba153ee-5c22-4252-a932-535e97e7b3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:39.985352 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba153ee-5c22-4252-a932-535e97e7b3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:39.985622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba153ee-5c22-4252-a932-535e97e7b3c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:39.988729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:39.993917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba153ee-5c22-4252-a932-535e97e7b3c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:39.995362 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba153ee-5c22-4252-a932-535e97e7b3c2 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:40.009318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-7ba153ee-5c22-4252-a932-535e97e7b3c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:40.009318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:40.009318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:40.009318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:40.009318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:40.009318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:40.009318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:40.009318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:40.009318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:40.009318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:40.009318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:40.009318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:40.009318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:40.009318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:40.009318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:40.010254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:40.010254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:40.010254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:40.010254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:40.010254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:40.010254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:40.010254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:40.010254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:40.010254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:40.010254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:40.010254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:40.010254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:40.010254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:40.010254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:40.010254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:40.010254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:40.010254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:40.016169 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:40.016169 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:40.016169 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:40.016169 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:40.016169 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:40.016169 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:40.016169 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:40.016169 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:40.016671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e1a1a-ca74-4d3d-8f95-febe9ec746ac None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:40.016747 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 325/646] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:40.016747 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba153ee-5c22-4252-a932-535e97e7b3c2 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:40.057084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16069cc1-1f1c-45fe-b21b-393dffe3136d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:40.057902 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16069cc1-1f1c-45fe-b21b-393dffe3136d None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:40.058186 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16069cc1-1f1c-45fe-b21b-393dffe3136d None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:40.058463 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16069cc1-1f1c-45fe-b21b-393dffe3136d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:40.068599 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16069cc1-1f1c-45fe-b21b-393dffe3136d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:40.070248 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16069cc1-1f1c-45fe-b21b-393dffe3136d None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:40.088126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-16069cc1-1f1c-45fe-b21b-393dffe3136d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:40.088126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:40.088126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:40.088126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:40.088126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:40.088126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:40.088126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:40.088126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:40.088126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:40.088126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:40.088126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:40.088126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:40.088126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:40.088126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:40.088126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:40.089149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:40.089149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:40.089149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:40.089149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:40.089149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:40.089149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:40.089149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:40.089149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:40.089149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:40.089149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:40.089149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:40.089149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:40.089149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:40.089149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:40.089149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:40.089149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:40.089149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:40.090028 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:40.090028 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:40.090028 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:40.090028 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:40.092907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 322/647] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:40.094296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16069cc1-1f1c-45fe-b21b-393dffe3136d None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:40.118473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba153ee-5c22-4252-a932-535e97e7b3c2 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:40.154019 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207b833-f9a8-4b41-9476-5d0c0343708c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:40.154805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207b833-f9a8-4b41-9476-5d0c0343708c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:40.155087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207b833-f9a8-4b41-9476-5d0c0343708c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:40.155606 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207b833-f9a8-4b41-9476-5d0c0343708c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:40.166463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d207b833-f9a8-4b41-9476-5d0c0343708c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:40.174376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-d207b833-f9a8-4b41-9476-5d0c0343708c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:40.174376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:40.174376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:40.174376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:40.174376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:40.174376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:40.174376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:40.174376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:40.174376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:40.174376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:40.174376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:40.174376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:40.174376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:40.174376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:40.174376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:40.174376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:40.175281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:40.175281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:40.175281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:40.175281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:40.175281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:40.175281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:40.175281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:40.175281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:40.175281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:40.175281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:40.175281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:40.175281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:40.175281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:40.175281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:40.175281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:40.175281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:40.176111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:40.176111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:40.176111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:40.176111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:40.176111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:40.176111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:40.176111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:40.176111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:40.176111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:40.176111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:40.176111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:40.176111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:40.176111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 326/648] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:27:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:40.177383 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207b833-f9a8-4b41-9476-5d0c0343708c None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:40.205171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:40.212452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16069cc1-1f1c-45fe-b21b-393dffe3136d None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:40.242019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:40.247498 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfede7-527d-4dfe-953e-2143a911d4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:40.248147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfede7-527d-4dfe-953e-2143a911d4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:40.248431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfede7-527d-4dfe-953e-2143a911d4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:40.248741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfede7-527d-4dfe-953e-2143a911d4d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:40.280859 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207b833-f9a8-4b41-9476-5d0c0343708c None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:40.320495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d89175-16be-4288-be3e-b917d3e85eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:40.321182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d89175-16be-4288-be3e-b917d3e85eac None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:40.321470 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d89175-16be-4288-be3e-b917d3e85eac None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:40.321748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d89175-16be-4288-be3e-b917d3e85eac None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:40.322538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d89175-16be-4288-be3e-b917d3e85eac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:40.322849 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d89175-16be-4288-be3e-b917d3e85eac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:40.324279 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d89175-16be-4288-be3e-b917d3e85eac None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:40.325956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cfede7-527d-4dfe-953e-2143a911d4d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:40.327395 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cfede7-527d-4dfe-953e-2143a911d4d3 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:40.337783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 327/649] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:40.338440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d89175-16be-4288-be3e-b917d3e85eac None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:40.425261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:40.440594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 323/650] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:40.441559 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d89175-16be-4288-be3e-b917d3e85eac None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:40.441796 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfede7-527d-4dfe-953e-2143a911d4d3 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:40.472993 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:40.481537 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e1ae5-6581-4ed0-a9f9-d4e3ad4d6da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:40.482368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e1ae5-6581-4ed0-a9f9-d4e3ad4d6da5 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:40.482654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e1ae5-6581-4ed0-a9f9-d4e3ad4d6da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:40.482929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e1ae5-6581-4ed0-a9f9-d4e3ad4d6da5 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:40.483658 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e1ae5-6581-4ed0-a9f9-d4e3ad4d6da5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:40.483961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e1ae5-6581-4ed0-a9f9-d4e3ad4d6da5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:40.485403 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e1ae5-6581-4ed0-a9f9-d4e3ad4d6da5 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:40.498678 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 328/651] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:40.499496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e1ae5-6581-4ed0-a9f9-d4e3ad4d6da5 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:40.550069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfede7-527d-4dfe-953e-2143a911d4d3 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:40.591007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929151e-a7da-43b9-8192-53e4ac1cca87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:40.592055 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929151e-a7da-43b9-8192-53e4ac1cca87 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:40.592401 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929151e-a7da-43b9-8192-53e4ac1cca87 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:40.592796 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929151e-a7da-43b9-8192-53e4ac1cca87 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:40.593609 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929151e-a7da-43b9-8192-53e4ac1cca87 None admin] RBAC: Adding query filter params (name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:40.593981 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929151e-a7da-43b9-8192-53e4ac1cca87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:40.595823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929151e-a7da-43b9-8192-53e4ac1cca87 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:40.608631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 324/652] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:27:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:40.608631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929151e-a7da-43b9-8192-53e4ac1cca87 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:40.611387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e1ae5-6581-4ed0-a9f9-d4e3ad4d6da5 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:40.646654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb0bd2-8bce-4726-b297-52706a86de2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:40.647365 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb0bd2-8bce-4726-b297-52706a86de2f None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:40.647650 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb0bd2-8bce-4726-b297-52706a86de2f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:40.648329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb0bd2-8bce-4726-b297-52706a86de2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:40.649193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb0bd2-8bce-4726-b297-52706a86de2f None admin] RBAC: Adding query filter params (role.id=698ff821321a43d494fe8940cdd08564, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=159cc8befc3a4d57bbd313832084db62) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:40.649505 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb0bd2-8bce-4726-b297-52706a86de2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:40.651599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb0bd2-8bce-4726-b297-52706a86de2f None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:40.664616 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 329/653] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:27:40 2022] GET /identity/v3/role_assignments?user.id=159cc8befc3a4d57bbd313832084db62&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=698ff821321a43d494fe8940cdd08564 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:40.664616 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb0bd2-8bce-4726-b297-52706a86de2f None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:40.680150 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:40.722364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:40.735715 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929151e-a7da-43b9-8192-53e4ac1cca87 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:40.783823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47765e9-ce5a-4bd9-b7b1-68b28b32481d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:40.784952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47765e9-ce5a-4bd9-b7b1-68b28b32481d None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:40.785310 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47765e9-ce5a-4bd9-b7b1-68b28b32481d None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:40.785995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47765e9-ce5a-4bd9-b7b1-68b28b32481d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:40.793628 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb0bd2-8bce-4726-b297-52706a86de2f None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:40.800451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47765e9-ce5a-4bd9-b7b1-68b28b32481d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:40.802562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47765e9-ce5a-4bd9-b7b1-68b28b32481d None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:40.825215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-a47765e9-ce5a-4bd9-b7b1-68b28b32481d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:40.825215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:40.825215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:40.825215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:40.825215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:40.825215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:40.825215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:40.825215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:40.825215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:40.825215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:40.825215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:40.825215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:40.825215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:40.825215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:40.825215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:40.826160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:40.826160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:40.826160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:40.826160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:40.826160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:40.826160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:40.826160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:40.826160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:40.826160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:40.826160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:40.826160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:40.826160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:40.826160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:40.826160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:40.826160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:40.826160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:40.826160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:40.827148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:40.827148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:40.827148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:40.827148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:40.827148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 325/654] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:40.828658 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47765e9-ce5a-4bd9-b7b1-68b28b32481d None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:40.847155 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b759d-dfa0-4aec-bd9a-72a730984953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:40.847964 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b759d-dfa0-4aec-bd9a-72a730984953 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:40.848714 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b759d-dfa0-4aec-bd9a-72a730984953 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:40.848998 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b759d-dfa0-4aec-bd9a-72a730984953 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:40.859707 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b759d-dfa0-4aec-bd9a-72a730984953 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:40.862601 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b759d-dfa0-4aec-bd9a-72a730984953 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:40.879395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-e09b759d-dfa0-4aec-bd9a-72a730984953 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 22:27:40.879395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:40.879395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:40.879395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:40.879395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:40.879395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:40.879395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:40.879395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:40.879395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:40.879395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:40.879395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:40.879395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:40.879395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:40.879395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:40.879395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:40.879395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:40.880557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:40.880557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:40.880557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:40.880557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:40.880557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:40.880557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:40.880557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:40.880557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:40.880557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:40.880557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:40.880557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:40.880557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:40.880557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:40.880557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:40.880557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:40.880557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:40.880557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:40.880557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:40.882173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:40.882173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:40.882173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:40.882173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:40.882173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:40.882173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:40.882173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:40.882173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:40.882173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 22:27:40.882173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:40.882173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 330/655] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:40.882173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b759d-dfa0-4aec-bd9a-72a730984953 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:40.947270 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47765e9-ce5a-4bd9-b7b1-68b28b32481d None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:40.963148 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:40.991232 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e6882-be61-4b18-8d16-35a6fb793a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:40.992253 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e6882-be61-4b18-8d16-35a6fb793a34 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:40.992546 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e6882-be61-4b18-8d16-35a6fb793a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:40.993003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e6882-be61-4b18-8d16-35a6fb793a34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:40.998198 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b759d-dfa0-4aec-bd9a-72a730984953 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:41.003736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047e6882-be61-4b18-8d16-35a6fb793a34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:41.005252 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047e6882-be61-4b18-8d16-35a6fb793a34 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:41.008157 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:41.024517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-047e6882-be61-4b18-8d16-35a6fb793a34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:41.024517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:41.024517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:41.024517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:41.024517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:41.024517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:41.024517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:41.024517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:41.024517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:41.024517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:41.024517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:41.024517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:41.024517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:41.024517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:41.024517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:41.025633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:41.025633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:41.025633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:41.025633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:41.025633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:41.025633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:41.025633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:41.025633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:41.025633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:41.025633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:41.025633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:41.025633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:41.025633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:41.025633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:41.025633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:41.025633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:41.025633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:41.026803 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:41.026803 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:41.026803 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:41.026803 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:41.029126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 326/656] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:41.031064 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fee32-2d5c-42fd-8980-ed31f2d42734 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:41.031356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fee32-2d5c-42fd-8980-ed31f2d42734 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:41.031789 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fee32-2d5c-42fd-8980-ed31f2d42734 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:41.033724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 327/657] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:41.034716 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fee32-2d5c-42fd-8980-ed31f2d42734 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:41.044510 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff00b3-a46b-426f-8862-718f991f0ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:41.045639 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff00b3-a46b-426f-8862-718f991f0ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:41.046128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff00b3-a46b-426f-8862-718f991f0ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:41.046604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff00b3-a46b-426f-8862-718f991f0ec6 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:41.047643 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff00b3-a46b-426f-8862-718f991f0ec6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:41.048163 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff00b3-a46b-426f-8862-718f991f0ec6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:41.049821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff00b3-a46b-426f-8862-718f991f0ec6 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:41.066629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 331/658] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:41.066629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff00b3-a46b-426f-8862-718f991f0ec6 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:41.152456 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fee32-2d5c-42fd-8980-ed31f2d42734 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:41.178928 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff00b3-a46b-426f-8862-718f991f0ec6 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:41.197449 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac16b5e-54cc-43b3-8739-1d9423aa3125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:41.198242 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac16b5e-54cc-43b3-8739-1d9423aa3125 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:41.198580 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac16b5e-54cc-43b3-8739-1d9423aa3125 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:41.198928 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac16b5e-54cc-43b3-8739-1d9423aa3125 None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:41.199881 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac16b5e-54cc-43b3-8739-1d9423aa3125 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:41.200263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac16b5e-54cc-43b3-8739-1d9423aa3125 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:41.201828 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac16b5e-54cc-43b3-8739-1d9423aa3125 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:41.217671 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:41.228604 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 328/659] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 23 22:27:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:41.231484 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1534377-0896-4983-adb2-0c5972412d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:41.231484 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1534377-0896-4983-adb2-0c5972412d59 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:41.231720 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963e698-e011-4766-96cc-cc52185ca930 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:41.231720 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963e698-e011-4766-96cc-cc52185ca930 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:41.231892 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1534377-0896-4983-adb2-0c5972412d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:41.231983 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963e698-e011-4766-96cc-cc52185ca930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:41.232078 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1534377-0896-4983-adb2-0c5972412d59 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:41.232078 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1534377-0896-4983-adb2-0c5972412d59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:41.232078 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1534377-0896-4983-adb2-0c5972412d59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:41.232078 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1534377-0896-4983-adb2-0c5972412d59 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:41.244507 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 332/660] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:41.244507 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1534377-0896-4983-adb2-0c5972412d59 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:41.249195 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-7963e698-e011-4766-96cc-cc52185ca930 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:41.250423 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-7963e698-e011-4766-96cc-cc52185ca930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:41.306880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:41.328769 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 329/661] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:41.328769 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963e698-e011-4766-96cc-cc52185ca930 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:41.361489 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1534377-0896-4983-adb2-0c5972412d59 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:41.400527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039425e3-6e73-4311-ac40-83f7e6a5eb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:41.401692 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039425e3-6e73-4311-ac40-83f7e6a5eb2a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:41.402179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039425e3-6e73-4311-ac40-83f7e6a5eb2a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:41.402663 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039425e3-6e73-4311-ac40-83f7e6a5eb2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:41.412001 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039425e3-6e73-4311-ac40-83f7e6a5eb2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:41.414131 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039425e3-6e73-4311-ac40-83f7e6a5eb2a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:41.426998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963e698-e011-4766-96cc-cc52185ca930 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:41.429469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-039425e3-6e73-4311-ac40-83f7e6a5eb2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:41.429469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:41.429469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:41.429469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:41.429469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:41.429469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:41.429469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:41.429469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:41.429469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:41.429469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:41.429469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:41.429469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:41.429469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:41.429469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:41.429469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:41.430542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:41.430542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:41.430542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:41.430542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:41.430542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:41.430542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:41.430542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:41.430542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:41.430542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:41.430542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:41.430542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:41.430542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:41.430542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:41.430542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:41.430542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:41.430542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:41.430542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:41.431538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:41.431538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:41.431538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:41.431538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:41.431538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:41.431538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:41.431538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:41.431538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:41.431538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 333/662] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:41.431538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039425e3-6e73-4311-ac40-83f7e6a5eb2a None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:41.467862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda071d-f3dc-45f0-b20c-afd412e91869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:41.468972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda071d-f3dc-45f0-b20c-afd412e91869 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:41.470241 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda071d-f3dc-45f0-b20c-afd412e91869 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:41.470736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda071d-f3dc-45f0-b20c-afd412e91869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:41.535813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039425e3-6e73-4311-ac40-83f7e6a5eb2a None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:41.541018 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda071d-f3dc-45f0-b20c-afd412e91869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:41.542877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda071d-f3dc-45f0-b20c-afd412e91869 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:41.548714 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:41.573242 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898517a7-1af7-4e9f-bd89-52cb384abf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:41.574039 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898517a7-1af7-4e9f-bd89-52cb384abf1b None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:41.574330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898517a7-1af7-4e9f-bd89-52cb384abf1b None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:41.574614 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898517a7-1af7-4e9f-bd89-52cb384abf1b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:41.581851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898517a7-1af7-4e9f-bd89-52cb384abf1b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:41.583350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898517a7-1af7-4e9f-bd89-52cb384abf1b None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:41.589972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:41.599229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-898517a7-1af7-4e9f-bd89-52cb384abf1b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 22:27:41.599229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:41.599229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:41.599229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:41.599229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:41.599229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:41.599229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:41.599229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:41.599229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:41.599229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:41.599229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:41.599229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:41.599229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:41.599229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:41.599229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:41.599229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:41.601215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:41.601215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:41.601215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:41.601215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:41.601215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:41.601215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:41.601215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:41.601215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:41.601215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:41.601215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:41.601215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:41.601215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:41.601215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:41.601215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:41.601215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:41.601215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:41.601215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:41.601215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:41.602421 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:41.602421 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:41.602421 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:41.602421 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:41.602421 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:41.602421 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:41.602421 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:41.602421 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:41.602421 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 22:27:41.602421 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:41.602421 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 334/663] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:27:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:41.602421 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898517a7-1af7-4e9f-bd89-52cb384abf1b None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:41.657696 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 330/664] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:41.657696 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda071d-f3dc-45f0-b20c-afd412e91869 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:41.712082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898517a7-1af7-4e9f-bd89-52cb384abf1b None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:41.722608 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:41.745723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:41.749792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b1fc6-75cf-400d-85e1-f5fdf74bfed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:41.750666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b1fc6-75cf-400d-85e1-f5fdf74bfed5 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:41.750956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b1fc6-75cf-400d-85e1-f5fdf74bfed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:41.751298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b1fc6-75cf-400d-85e1-f5fdf74bfed5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:41.762163 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b1fc6-75cf-400d-85e1-f5fdf74bfed5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:41.765420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda071d-f3dc-45f0-b20c-afd412e91869 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:41.769894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-0f7b1fc6-75cf-400d-85e1-f5fdf74bfed5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:41.769894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:41.769894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:41.769894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:41.769894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:41.769894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:41.769894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:41.769894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:41.769894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:41.769894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:41.769894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:41.769894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:41.769894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:41.769894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:41.769894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:41.769894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:41.771219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:41.771219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:41.771219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:41.771219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:41.771219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:41.771219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:41.771219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:41.771219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:41.771219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:41.771219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:41.771219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:41.771219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:41.771219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:41.771219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:41.771219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:41.771219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:41.772750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:41.772750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:41.772750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:41.772750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:41.772750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:41.772750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:41.772750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:41.772750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:41.772750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:41.772750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:41.772750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:41.772750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:41.772750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 335/665] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:27:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:41.772750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b1fc6-75cf-400d-85e1-f5fdf74bfed5 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:41.810665 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbc374-e41b-45d9-8a55-7233c623ab0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:41.811254 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbc374-e41b-45d9-8a55-7233c623ab0a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:41.811541 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbc374-e41b-45d9-8a55-7233c623ab0a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:41.811821 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbc374-e41b-45d9-8a55-7233c623ab0a None admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:41.812636 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbc374-e41b-45d9-8a55-7233c623ab0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:41.812951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbc374-e41b-45d9-8a55-7233c623ab0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:41.814758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbc374-e41b-45d9-8a55-7233c623ab0a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:41.834143 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 331/666] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:41.834143 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbc374-e41b-45d9-8a55-7233c623ab0a None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:41.880624 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b1fc6-75cf-400d-85e1-f5fdf74bfed5 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:41.917199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435850be-0540-4d5a-a1d8-f05713892996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:41.917870 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435850be-0540-4d5a-a1d8-f05713892996 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:41.918164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435850be-0540-4d5a-a1d8-f05713892996 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:41.918436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435850be-0540-4d5a-a1d8-f05713892996 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:41.919274 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435850be-0540-4d5a-a1d8-f05713892996 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:41.919586 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435850be-0540-4d5a-a1d8-f05713892996 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:41.921041 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435850be-0540-4d5a-a1d8-f05713892996 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:41.943187 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 336/667] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Jun 23 22:27:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:41.943187 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435850be-0540-4d5a-a1d8-f05713892996 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:41.943489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbc374-e41b-45d9-8a55-7233c623ab0a None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:41.973069 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:41.978577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a5899-39e1-4da7-a2b9-0ec5e50e7f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:41.979383 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a5899-39e1-4da7-a2b9-0ec5e50e7f3d None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:41.985451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a5899-39e1-4da7-a2b9-0ec5e50e7f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:41.985451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a5899-39e1-4da7-a2b9-0ec5e50e7f3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:41.990450 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a5899-39e1-4da7-a2b9-0ec5e50e7f3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:41.991931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a5899-39e1-4da7-a2b9-0ec5e50e7f3d None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:41.994735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:42.010420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-795a5899-39e1-4da7-a2b9-0ec5e50e7f3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:42.010420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:42.010420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:42.010420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:42.010420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:42.010420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:42.010420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:42.010420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:42.010420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:42.010420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:42.010420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:42.010420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:42.010420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:42.010420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:42.010420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:42.011644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:42.011644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:42.011644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:42.011644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:42.011644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:42.011644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:42.011644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:42.011644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:42.011644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:42.011644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:42.011644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:42.011644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:42.011644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:42.011644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:42.011644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:42.011644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:42.011644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:42.012551 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:42.012551 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:42.012551 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:42.012551 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:42.012551 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 332/668] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:42.012883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a5899-39e1-4da7-a2b9-0ec5e50e7f3d None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:42.051765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435850be-0540-4d5a-a1d8-f05713892996 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:42.087230 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92872b-1a1e-4127-a0bf-b4ad2d49314a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:42.087828 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92872b-1a1e-4127-a0bf-b4ad2d49314a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:42.088114 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92872b-1a1e-4127-a0bf-b4ad2d49314a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:42.088392 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92872b-1a1e-4127-a0bf-b4ad2d49314a None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:42.089186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92872b-1a1e-4127-a0bf-b4ad2d49314a None admin] RBAC: Adding query filter params (name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:42.089498 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92872b-1a1e-4127-a0bf-b4ad2d49314a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:42.090861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92872b-1a1e-4127-a0bf-b4ad2d49314a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:42.099719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 337/669] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:27:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:42.100959 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92872b-1a1e-4127-a0bf-b4ad2d49314a None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:42.122685 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a5899-39e1-4da7-a2b9-0ec5e50e7f3d None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:42.160776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b3cd2-a1d0-4208-a1f8-f086dea27ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:42.161435 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b3cd2-a1d0-4208-a1f8-f086dea27ade None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:42.161722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b3cd2-a1d0-4208-a1f8-f086dea27ade None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:42.162000 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b3cd2-a1d0-4208-a1f8-f086dea27ade None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:42.162855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b3cd2-a1d0-4208-a1f8-f086dea27ade None admin] RBAC: Adding query filter params (role.id=698ff821321a43d494fe8940cdd08564, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=5d1911bfac984e058b0b11bb6400e782) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:42.163169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b3cd2-a1d0-4208-a1f8-f086dea27ade None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:42.164681 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b3cd2-a1d0-4208-a1f8-f086dea27ade None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:42.180605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 333/670] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:27:42 2022] GET /identity/v3/role_assignments?user.id=5d1911bfac984e058b0b11bb6400e782&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=698ff821321a43d494fe8940cdd08564 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:42.180605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b3cd2-a1d0-4208-a1f8-f086dea27ade None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:42.222813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:42.235485 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92872b-1a1e-4127-a0bf-b4ad2d49314a None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:42.248229 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:42.289370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f67f40-dd4b-4e09-b6bd-38071c7e221d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:42.290685 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f67f40-dd4b-4e09-b6bd-38071c7e221d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:42.290991 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f67f40-dd4b-4e09-b6bd-38071c7e221d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:42.291273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f67f40-dd4b-4e09-b6bd-38071c7e221d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:42.306771 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b3cd2-a1d0-4208-a1f8-f086dea27ade None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:42.349197 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3afe56-d225-453a-825d-a02f8da0e1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:42.350271 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3afe56-d225-453a-825d-a02f8da0e1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:42.350823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3afe56-d225-453a-825d-a02f8da0e1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:42.351300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3afe56-d225-453a-825d-a02f8da0e1a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:42.352402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3afe56-d225-453a-825d-a02f8da0e1a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:42.352765 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3afe56-d225-453a-825d-a02f8da0e1a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:42.354376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3afe56-d225-453a-825d-a02f8da0e1a3 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:42.373442 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 334/671] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:42.373442 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3afe56-d225-453a-825d-a02f8da0e1a3 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:42.390140 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f67f40-dd4b-4e09-b6bd-38071c7e221d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:42.391888 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f67f40-dd4b-4e09-b6bd-38071c7e221d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:42.486709 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3afe56-d225-453a-825d-a02f8da0e1a3 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:42.504625 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:42.509485 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:42.518758 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 338/672] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:42.521088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f67f40-dd4b-4e09-b6bd-38071c7e221d None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:42.530070 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ad7de-1d97-4c62-b910-927132ba0cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:42.530885 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ad7de-1d97-4c62-b910-927132ba0cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:42.531169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ad7de-1d97-4c62-b910-927132ba0cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:42.531449 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ad7de-1d97-4c62-b910-927132ba0cf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:42.545724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ad7de-1d97-4c62-b910-927132ba0cf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:42.547208 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ad7de-1d97-4c62-b910-927132ba0cf9 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:42.570830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-c20ad7de-1d97-4c62-b910-927132ba0cf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:42.570830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:42.570830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:42.570830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:42.570830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:42.570830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:42.570830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:42.570830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:42.570830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:42.570830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:42.570830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:42.570830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:42.570830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:42.570830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:42.570830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:42.571838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:42.571838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:42.571838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:42.571838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:42.571838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:42.571838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:42.571838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:42.571838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:42.571838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:42.571838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:42.571838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:42.571838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:42.571838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:42.571838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:42.571838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:42.571838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:42.571838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:42.572847 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:42.572847 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:42.572847 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:42.572847 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:42.572847 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 335/673] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:42.573614 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ad7de-1d97-4c62-b910-927132ba0cf9 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:42.643662 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f67f40-dd4b-4e09-b6bd-38071c7e221d None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:42.691951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ad7de-1d97-4c62-b910-927132ba0cf9 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:42.693342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3e74b-9519-4909-b89f-44bed8e52921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:42.695666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3e74b-9519-4909-b89f-44bed8e52921 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:42.695958 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3e74b-9519-4909-b89f-44bed8e52921 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:42.696237 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3e74b-9519-4909-b89f-44bed8e52921 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:42.705043 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3e74b-9519-4909-b89f-44bed8e52921 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:42.706778 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3e74b-9519-4909-b89f-44bed8e52921 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:42.727520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-85f3e74b-9519-4909-b89f-44bed8e52921 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:42.727520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:42.727520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:42.727520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:42.727520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:42.727520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:42.727520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:42.727520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:42.727520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:42.727520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:42.727520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:42.727520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:42.727520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:42.727520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:42.727520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:42.729533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:42.729533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:42.729533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:42.729533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:42.729533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:42.729533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:42.729533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:42.729533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:42.729533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:42.729533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:42.729533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:42.729533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:42.729533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:42.729533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:42.729533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:42.729533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:42.729533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:42.730792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:42.730792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:42.730792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:42.730792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:42.730792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:42.730792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:42.730792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:42.730792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:42.736596 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 339/674] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:42.736596 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3e74b-9519-4909-b89f-44bed8e52921 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:42.736985 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f011db-214e-46fc-9f37-8d879f953558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:42.736985 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f011db-214e-46fc-9f37-8d879f953558 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:42.736985 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f011db-214e-46fc-9f37-8d879f953558 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:42.736985 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f011db-214e-46fc-9f37-8d879f953558 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:42.747606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f011db-214e-46fc-9f37-8d879f953558 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:42.749108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f011db-214e-46fc-9f37-8d879f953558 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:42.768196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-13f011db-214e-46fc-9f37-8d879f953558 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:42.768196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:42.768196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:42.768196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:42.768196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:42.768196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:42.768196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:42.768196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:42.768196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:42.768196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:42.768196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:42.768196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:42.768196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:42.768196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:42.768196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:42.770308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:42.770308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:42.770308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:42.770308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:42.770308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:42.770308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:42.770308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:42.770308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:42.770308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:42.770308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:42.770308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:42.770308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:42.770308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:42.770308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:42.770308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:42.770308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:42.770308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:42.773187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:42.773187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:42.773187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:42.773187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:42.776626 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 336/675] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:42.776626 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f011db-214e-46fc-9f37-8d879f953558 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:42.799476 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:42.805714 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:42.855311 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3e74b-9519-4909-b89f-44bed8e52921 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:42.885112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f011db-214e-46fc-9f37-8d879f953558 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:42.890605 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91362827-0f49-4887-8870-6aa4f64521c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:42.891280 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91362827-0f49-4887-8870-6aa4f64521c0 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:42.891570 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91362827-0f49-4887-8870-6aa4f64521c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:42.891854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91362827-0f49-4887-8870-6aa4f64521c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:42.892610 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91362827-0f49-4887-8870-6aa4f64521c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:42.892921 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91362827-0f49-4887-8870-6aa4f64521c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:42.894345 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91362827-0f49-4887-8870-6aa4f64521c0 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:42.911348 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 340/676] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:42.912066 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91362827-0f49-4887-8870-6aa4f64521c0 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:42.922173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c74f6-609c-47fd-9ab5-364d751f5381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:42.923085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c74f6-609c-47fd-9ab5-364d751f5381 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:42.923085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c74f6-609c-47fd-9ab5-364d751f5381 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:42.923373 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c74f6-609c-47fd-9ab5-364d751f5381 None admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:42.924145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c74f6-609c-47fd-9ab5-364d751f5381 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:42.924452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c74f6-609c-47fd-9ab5-364d751f5381 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:42.925904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c74f6-609c-47fd-9ab5-364d751f5381 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:42.942933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 337/677] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:42.942933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a2b41-b015-4b2f-b997-840cc8e691ca None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:42.942933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a2b41-b015-4b2f-b997-840cc8e691ca None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:42.942933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a2b41-b015-4b2f-b997-840cc8e691ca None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:42.942933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 338/678] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:42.942933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a2b41-b015-4b2f-b997-840cc8e691ca None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:43.023887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91362827-0f49-4887-8870-6aa4f64521c0 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:43.034137 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:43.034720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:43.052195 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a2b41-b015-4b2f-b997-840cc8e691ca None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:43.059368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72399d0-859a-440a-837d-40e4a4c32078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:43.060052 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72399d0-859a-440a-837d-40e4a4c32078 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:43.060338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72399d0-859a-440a-837d-40e4a4c32078 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:43.060645 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72399d0-859a-440a-837d-40e4a4c32078 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:43.061385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72399d0-859a-440a-837d-40e4a4c32078 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:43.061705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72399d0-859a-440a-837d-40e4a4c32078 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:43.063137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72399d0-859a-440a-837d-40e4a4c32078 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:43.074747 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 341/679] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:43.079689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72399d0-859a-440a-837d-40e4a4c32078 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:43.088618 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c10ea0-d850-4db2-a20c-78e5634b0e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:43.089406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c10ea0-d850-4db2-a20c-78e5634b0e4a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:43.089689 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c10ea0-d850-4db2-a20c-78e5634b0e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:43.089956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c10ea0-d850-4db2-a20c-78e5634b0e4a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:43.097410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c10ea0-d850-4db2-a20c-78e5634b0e4a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:43.098859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c10ea0-d850-4db2-a20c-78e5634b0e4a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:43.113524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-28c10ea0-d850-4db2-a20c-78e5634b0e4a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:27:43.113524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:43.113524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:43.113524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:43.113524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:43.113524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:43.113524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:43.113524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:43.113524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:43.113524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:43.113524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:43.113524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:43.113524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:43.113524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:43.113524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:43.113524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:43.114451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:43.114451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:43.114451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:43.114451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:43.114451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:43.114451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:43.114451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:43.114451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:43.114451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:43.114451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:43.114451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:43.114451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:43.114451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:43.114451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:43.114451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:43.114451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:43.114451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:43.114451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:43.115406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:43.115406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:43.115406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:43.115406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:43.115406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:43.115406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:43.115406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:43.115406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:43.115406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:27:43.115406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:43.115406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 339/680] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:27:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:43.117388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd1e3c-02d0-4fc9-8463-a001b2c8af4a None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:43.117388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd1e3c-02d0-4fc9-8463-a001b2c8af4a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:43.117388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd1e3c-02d0-4fc9-8463-a001b2c8af4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:43.132542 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-b7fd1e3c-02d0-4fc9-8463-a001b2c8af4a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:43.133727 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-b7fd1e3c-02d0-4fc9-8463-a001b2c8af4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:43.195717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72399d0-859a-440a-837d-40e4a4c32078 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:43.210896 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 340/681] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:43.211817 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd1e3c-02d0-4fc9-8463-a001b2c8af4a None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:43.241629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2facf49-8f4a-4f57-8100-5f5a9256eecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:43.242377 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2facf49-8f4a-4f57-8100-5f5a9256eecf None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:43.242666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2facf49-8f4a-4f57-8100-5f5a9256eecf None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:43.242944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2facf49-8f4a-4f57-8100-5f5a9256eecf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:43.243773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2facf49-8f4a-4f57-8100-5f5a9256eecf None admin] RBAC: Adding query filter params (role.id=698ff821321a43d494fe8940cdd08564, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=983801bb18924fe4b3150693765eb13b) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:43.244082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2facf49-8f4a-4f57-8100-5f5a9256eecf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:43.245541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2facf49-8f4a-4f57-8100-5f5a9256eecf None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:43.251816 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 342/682] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:27:43 2022] GET /identity/v3/role_assignments?user.id=983801bb18924fe4b3150693765eb13b&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=698ff821321a43d494fe8940cdd08564 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:43.252615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2facf49-8f4a-4f57-8100-5f5a9256eecf None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:43.291133 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:43.305917 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:43.337658 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd1e3c-02d0-4fc9-8463-a001b2c8af4a None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:43.367082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2facf49-8f4a-4f57-8100-5f5a9256eecf None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:43.381151 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d866954-c753-4be2-9d53-2f02c8c55c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:43.384390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d866954-c753-4be2-9d53-2f02c8c55c13 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:43.385012 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d866954-c753-4be2-9d53-2f02c8c55c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:43.390575 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d866954-c753-4be2-9d53-2f02c8c55c13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:43.394644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d866954-c753-4be2-9d53-2f02c8c55c13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:43.396447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d866954-c753-4be2-9d53-2f02c8c55c13 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:43.406500 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd9fc8-ae6a-4d1d-b505-b6638bb5084a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:43.407186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd9fc8-ae6a-4d1d-b505-b6638bb5084a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:43.407461 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd9fc8-ae6a-4d1d-b505-b6638bb5084a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:43.407741 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd9fc8-ae6a-4d1d-b505-b6638bb5084a None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:43.408638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd9fc8-ae6a-4d1d-b505-b6638bb5084a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:43.408954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd9fc8-ae6a-4d1d-b505-b6638bb5084a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:43.410404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd9fc8-ae6a-4d1d-b505-b6638bb5084a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:43.416874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-9d866954-c753-4be2-9d53-2f02c8c55c13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:43.416874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:43.416874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:43.416874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:43.416874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:43.416874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:43.416874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:43.416874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:43.416874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:43.416874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:43.416874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:43.416874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:43.416874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:43.416874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:43.416874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:43.417822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:43.417822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:43.417822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:43.417822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:43.417822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:43.417822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:43.417822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:43.417822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:43.417822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:43.417822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:43.417822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:43.417822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:43.417822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:43.417822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:43.417822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:43.417822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:43.417822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:43.418713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:43.418713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:43.418713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:43.418713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:43.418713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:43.418713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:43.418713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:43.418713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:43.421373 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 341/683] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:43.422627 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d866954-c753-4be2-9d53-2f02c8c55c13 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:43.441201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 343/684] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 23 22:27:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:43.442083 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd9fc8-ae6a-4d1d-b505-b6638bb5084a None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:43.552440 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d866954-c753-4be2-9d53-2f02c8c55c13 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:43.554708 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:43.561476 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd9fc8-ae6a-4d1d-b505-b6638bb5084a None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:43.598663 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb4299-3d56-4f70-833c-2dddf323f76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:43.599082 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d84a8-26db-4fe2-a5e4-6725f4edf002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:43.599784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d84a8-26db-4fe2-a5e4-6725f4edf002 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:43.600105 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d84a8-26db-4fe2-a5e4-6725f4edf002 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:43.600394 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d84a8-26db-4fe2-a5e4-6725f4edf002 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:43.600666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb4299-3d56-4f70-833c-2dddf323f76d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:43.601311 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb4299-3d56-4f70-833c-2dddf323f76d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:43.601789 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb4299-3d56-4f70-833c-2dddf323f76d None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:43.602834 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb4299-3d56-4f70-833c-2dddf323f76d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:43.603348 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb4299-3d56-4f70-833c-2dddf323f76d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:43.605065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb4299-3d56-4f70-833c-2dddf323f76d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:43.611760 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d84a8-26db-4fe2-a5e4-6725f4edf002 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:43.620069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-be9d84a8-26db-4fe2-a5e4-6725f4edf002 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:27:43.620069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:43.620069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:43.620069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:43.620069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:43.620069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:43.620069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:43.620069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:43.620069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:43.620069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:43.620069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:43.620069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:43.620069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:43.620069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:43.620069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:43.620069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:43.621069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:43.621069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:43.621069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:43.621069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:43.621069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:43.621069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:43.621069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:43.621069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:43.621069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:43.621069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:43.621069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:43.621069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:43.621069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:43.621069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:43.621069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:43.621069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:43.621959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:43.621959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:43.621959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:43.621959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:43.621959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:43.621959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:43.621959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:43.621959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:43.621959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:43.621959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:43.621959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:27:43.621959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:43.622635 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 344/685] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:43.623202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb4299-3d56-4f70-833c-2dddf323f76d None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:43.625025 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 342/686] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:27:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:43.625599 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:43.636635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d84a8-26db-4fe2-a5e4-6725f4edf002 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:43.752129 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d84a8-26db-4fe2-a5e4-6725f4edf002 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:43.761098 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb4299-3d56-4f70-833c-2dddf323f76d None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:43.787087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7f0a9-3741-4f0d-9db0-5e9f4720dc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:43.788620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7f0a9-3741-4f0d-9db0-5e9f4720dc20 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:43.789050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7f0a9-3741-4f0d-9db0-5e9f4720dc20 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:43.789514 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7f0a9-3741-4f0d-9db0-5e9f4720dc20 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:43.790398 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7f0a9-3741-4f0d-9db0-5e9f4720dc20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:43.790882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7f0a9-3741-4f0d-9db0-5e9f4720dc20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:43.792755 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7f0a9-3741-4f0d-9db0-5e9f4720dc20 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:43.799724 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e01ddd-e066-4b62-ab0c-209cd7c078e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:43.800394 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e01ddd-e066-4b62-ab0c-209cd7c078e0 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:43.800713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e01ddd-e066-4b62-ab0c-209cd7c078e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:43.801006 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e01ddd-e066-4b62-ab0c-209cd7c078e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:43.807312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 343/687] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:43.808387 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7f0a9-3741-4f0d-9db0-5e9f4720dc20 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:43.825944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:43.874168 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:43.890567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e01ddd-e066-4b62-ab0c-209cd7c078e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:43.892946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e01ddd-e066-4b62-ab0c-209cd7c078e0 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:43.932795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7f0a9-3741-4f0d-9db0-5e9f4720dc20 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:43.970743 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28ffab-d583-474b-a97f-8ab7f7ad3f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:43.971406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28ffab-d583-474b-a97f-8ab7f7ad3f67 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:43.971754 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28ffab-d583-474b-a97f-8ab7f7ad3f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:43.972037 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28ffab-d583-474b-a97f-8ab7f7ad3f67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:43.972973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28ffab-d583-474b-a97f-8ab7f7ad3f67 None admin] RBAC: Adding query filter params (role.id=698ff821321a43d494fe8940cdd08564, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=ec7412a9a3dc46369fe7b4771e3437de) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:43.973293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28ffab-d583-474b-a97f-8ab7f7ad3f67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:43.974745 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28ffab-d583-474b-a97f-8ab7f7ad3f67 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:43.988608 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 344/688] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:27:43 2022] GET /identity/v3/role_assignments?user.id=ec7412a9a3dc46369fe7b4771e3437de&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=698ff821321a43d494fe8940cdd08564 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:43.988608 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28ffab-d583-474b-a97f-8ab7f7ad3f67 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:44.025481 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 345/689] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:44.031143 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e01ddd-e066-4b62-ab0c-209cd7c078e0 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:44.099677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28ffab-d583-474b-a97f-8ab7f7ad3f67 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:44.122869 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:44.124450 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:44.140895 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e6041-536b-4e03-8ae0-e11adccd1af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:44.142043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e6041-536b-4e03-8ae0-e11adccd1af1 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:44.142490 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e6041-536b-4e03-8ae0-e11adccd1af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:44.143360 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e6041-536b-4e03-8ae0-e11adccd1af1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:44.158351 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e6041-536b-4e03-8ae0-e11adccd1af1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:44.160404 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e6041-536b-4e03-8ae0-e11adccd1af1 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:44.185597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-015e6041-536b-4e03-8ae0-e11adccd1af1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:44.185597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:44.185597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:44.185597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:44.185597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:44.185597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:44.185597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:44.185597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:44.185597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:44.185597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:44.185597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:44.185597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:44.185597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:44.185597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:44.185597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:44.186763 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:44.186763 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:44.186763 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:44.186763 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:44.186763 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:44.186763 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:44.186763 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:44.186763 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:44.186763 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:44.186763 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:44.186763 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:44.186763 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:44.186763 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:44.186763 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:44.186763 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:44.186763 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:44.186763 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:44.187848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:44.187848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:44.187848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:44.187848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:44.192140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 345/690] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:44.197603 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e6041-536b-4e03-8ae0-e11adccd1af1 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:44.248297 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e01ddd-e066-4b62-ab0c-209cd7c078e0 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:44.320251 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357172a-11ef-4c2c-a009-62d4b09906d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:44.321109 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357172a-11ef-4c2c-a009-62d4b09906d3 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:44.321425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357172a-11ef-4c2c-a009-62d4b09906d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:44.321667 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357172a-11ef-4c2c-a009-62d4b09906d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:44.333792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357172a-11ef-4c2c-a009-62d4b09906d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:44.336065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357172a-11ef-4c2c-a009-62d4b09906d3 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:44.360054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-6357172a-11ef-4c2c-a009-62d4b09906d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:44.360054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:44.360054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:44.360054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:44.360054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:44.360054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:44.360054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:44.360054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:44.360054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:44.360054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:44.360054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:44.360054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:44.360054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:44.360054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:44.360054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:44.361182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:44.361182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:44.361182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:44.361182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:44.361182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:44.361182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:44.361182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:44.361182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:44.361182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:44.361182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:44.361182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:44.361182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:44.361182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:44.361182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:44.361182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:44.361182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:44.361182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:44.362324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:44.362324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:44.362324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:44.362324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:44.364719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 346/691] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:27:44.368164 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e6041-536b-4e03-8ae0-e11adccd1af1 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:44.371352 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357172a-11ef-4c2c-a009-62d4b09906d3 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:44.410845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc1010-50b7-465e-858d-d7658712a6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:44.411523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc1010-50b7-465e-858d-d7658712a6bc None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:44.411809 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc1010-50b7-465e-858d-d7658712a6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:44.412088 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc1010-50b7-465e-858d-d7658712a6bc None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:44.412880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc1010-50b7-465e-858d-d7658712a6bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:44.413196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc1010-50b7-465e-858d-d7658712a6bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:44.414904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc1010-50b7-465e-858d-d7658712a6bc None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:44.428975 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 346/692] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:27:44.430563 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801c51e-7cbc-48b7-b048-c7f78135c1e0 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:44.430859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801c51e-7cbc-48b7-b048-c7f78135c1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:44.431137 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801c51e-7cbc-48b7-b048-c7f78135c1e0 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:44.432705 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 347/693] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:44.435006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801c51e-7cbc-48b7-b048-c7f78135c1e0 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:44.476946 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:44.534312 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357172a-11ef-4c2c-a009-62d4b09906d3 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:44.538665 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:44.576696 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455c6a4-1d1c-40b1-b3f2-4f7ecf3b1c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:44.578081 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455c6a4-1d1c-40b1-b3f2-4f7ecf3b1c58 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:44.578482 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455c6a4-1d1c-40b1-b3f2-4f7ecf3b1c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:44.578850 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801c51e-7cbc-48b7-b048-c7f78135c1e0 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:44.578984 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455c6a4-1d1c-40b1-b3f2-4f7ecf3b1c58 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:44.579774 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455c6a4-1d1c-40b1-b3f2-4f7ecf3b1c58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:44.580178 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455c6a4-1d1c-40b1-b3f2-4f7ecf3b1c58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:44.582491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455c6a4-1d1c-40b1-b3f2-4f7ecf3b1c58 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:44.594649 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 347/694] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:44.596145 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22a6cd-f85f-47be-8dc6-152547b5fb8d None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:44.596516 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22a6cd-f85f-47be-8dc6-152547b5fb8d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:44.596913 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22a6cd-f85f-47be-8dc6-152547b5fb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:44.614837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-eb22a6cd-f85f-47be-8dc6-152547b5fb8d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:44.616905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-eb22a6cd-f85f-47be-8dc6-152547b5fb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:44.626281 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca1eb5-8708-4206-8ddf-b9bbc970c80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:44.627332 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca1eb5-8708-4206-8ddf-b9bbc970c80a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:44.627766 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca1eb5-8708-4206-8ddf-b9bbc970c80a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:44.628665 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca1eb5-8708-4206-8ddf-b9bbc970c80a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:44.638852 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca1eb5-8708-4206-8ddf-b9bbc970c80a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:44.641082 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca1eb5-8708-4206-8ddf-b9bbc970c80a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:44.661007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-a5ca1eb5-8708-4206-8ddf-b9bbc970c80a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:27:44.661007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:44.661007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:44.661007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:44.661007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:44.661007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:44.661007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:44.661007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:44.661007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:44.661007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:44.661007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:44.661007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:44.661007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:44.661007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:44.661007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:44.661007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:44.662173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:44.662173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:44.662173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:44.662173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:44.662173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:44.662173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:44.662173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:44.662173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:44.662173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:44.662173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:44.662173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:44.662173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:44.662173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:44.662173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:44.662173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:44.662173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:44.662173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:44.662173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:44.663899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:44.663899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:44.663899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:44.663899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:44.663899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:44.663899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:44.663899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:44.663899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:44.663899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:27:44.663899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:44.666960 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 348/695] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 23 22:27:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:44.668417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca1eb5-8708-4206-8ddf-b9bbc970c80a None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:44.697592 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 348/696] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:44.698746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22a6cd-f85f-47be-8dc6-152547b5fb8d None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:44.791869 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:44.802188 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca1eb5-8708-4206-8ddf-b9bbc970c80a None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:44.809659 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:44.816739 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22a6cd-f85f-47be-8dc6-152547b5fb8d None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:44.848198 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acac2b-c51e-4b59-9061-d30e8313152f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:44.849358 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acac2b-c51e-4b59-9061-d30e8313152f None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:44.849622 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acac2b-c51e-4b59-9061-d30e8313152f None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:44.849912 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acac2b-c51e-4b59-9061-d30e8313152f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:44.854787 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bceac-9c0a-48e2-80bc-fe37be91c408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:44.855715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bceac-9c0a-48e2-80bc-fe37be91c408 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:44.856431 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bceac-9c0a-48e2-80bc-fe37be91c408 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:44.856949 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bceac-9c0a-48e2-80bc-fe37be91c408 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:44.858133 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bceac-9c0a-48e2-80bc-fe37be91c408 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:44.858771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bceac-9c0a-48e2-80bc-fe37be91c408 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:44.859286 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4acac2b-c51e-4b59-9061-d30e8313152f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:44.860704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bceac-9c0a-48e2-80bc-fe37be91c408 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:44.860812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4acac2b-c51e-4b59-9061-d30e8313152f None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:44.876616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-f4acac2b-c51e-4b59-9061-d30e8313152f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:44.876616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:44.876616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:44.876616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:44.876616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:44.876616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:44.876616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:44.876616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:44.876616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:44.876616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:44.876616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:44.876616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:44.876616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:44.876616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:44.876616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:44.877528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:44.877528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:44.877528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:44.877528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:44.877528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:44.877528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:44.877528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:44.877528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:44.877528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:44.877528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:44.877528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:44.877528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:44.877528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:44.877528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:44.877528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:44.877528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:44.877528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:44.878436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:44.878436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:44.878436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:44.878436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:44.878436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:44.878436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:44.878436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:44.878436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:44.880897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 349/697] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:44.882680 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acac2b-c51e-4b59-9061-d30e8313152f None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:44.883080 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 349/698] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 23 22:27:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:44.889552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bceac-9c0a-48e2-80bc-fe37be91c408 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:45.001775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acac2b-c51e-4b59-9061-d30e8313152f None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:45.005793 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bceac-9c0a-48e2-80bc-fe37be91c408 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:45.040487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b463e-cf4d-42ab-a755-206250570394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:45.041686 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b463e-cf4d-42ab-a755-206250570394 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:45.041979 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b463e-cf4d-42ab-a755-206250570394 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:45.042264 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b463e-cf4d-42ab-a755-206250570394 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:45.042927 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b463e-cf4d-42ab-a755-206250570394 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:45.044318 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b463e-cf4d-42ab-a755-206250570394 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:45.052674 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc6aac-7399-40e8-bdf3-e2302d0f76b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:45.054458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc6aac-7399-40e8-bdf3-e2302d0f76b9 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:45.054997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc6aac-7399-40e8-bdf3-e2302d0f76b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:45.055483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc6aac-7399-40e8-bdf3-e2302d0f76b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:45.056531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc6aac-7399-40e8-bdf3-e2302d0f76b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:45.057134 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc6aac-7399-40e8-bdf3-e2302d0f76b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:45.058751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc6aac-7399-40e8-bdf3-e2302d0f76b9 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:45.059451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-5f1b463e-cf4d-42ab-a755-206250570394 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 22:27:45.059451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:45.059451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:27:45.059451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:27:45.059451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:27:45.059451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:27:45.059451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:27:45.059451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:27:45.059451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 22:27:45.059451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:45.059451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:27:45.059451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:45.059451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:45.059451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:45.059451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:45.059451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:45.060384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:45.060384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:45.060384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:45.060384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:45.060384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:45.060384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:45.060384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:45.060384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:27:45.060384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:27:45.060384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:27:45.060384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:27:45.060384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:45.060384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:45.060384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:45.060384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:45.060384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:45.061284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:45.061284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:45.061284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:45.061284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:45.061284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:45.061284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:45.061284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:45.061284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:45.061284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:45.061284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:45.061284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:45.061284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:27:45.061284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:27:45.061284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 22:27:45.061284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:45.065696 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 350/699] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Jun 23 22:27:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:45.066536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b463e-cf4d-42ab-a755-206250570394 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:45.077888 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 350/700] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:45.079458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc6aac-7399-40e8-bdf3-e2302d0f76b9 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:45.087325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:45.101711 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:45.175049 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b463e-cf4d-42ab-a755-206250570394 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:45.196088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc6aac-7399-40e8-bdf3-e2302d0f76b9 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:45.212666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8ca7d-d88c-4f95-9583-16afbf1ecaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:45.213231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8ca7d-d88c-4f95-9583-16afbf1ecaf9 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:45.213494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8ca7d-d88c-4f95-9583-16afbf1ecaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:45.213779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8ca7d-d88c-4f95-9583-16afbf1ecaf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:45.234965 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d98f0-9ee4-4c75-98ff-6e65061be078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:45.236088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d98f0-9ee4-4c75-98ff-6e65061be078 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:45.236587 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d98f0-9ee4-4c75-98ff-6e65061be078 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:45.237066 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d98f0-9ee4-4c75-98ff-6e65061be078 None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:45.237991 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d98f0-9ee4-4c75-98ff-6e65061be078 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:45.238629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d98f0-9ee4-4c75-98ff-6e65061be078 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:45.240288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d98f0-9ee4-4c75-98ff-6e65061be078 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:45.246040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 351/701] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 23 22:27:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:45.247100 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d98f0-9ee4-4c75-98ff-6e65061be078 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:45.301377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8ca7d-d88c-4f95-9583-16afbf1ecaf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:45.302924 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8ca7d-d88c-4f95-9583-16afbf1ecaf9 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:45.321769 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:45.337011 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:45.364141 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d98f0-9ee4-4c75-98ff-6e65061be078 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:45.404052 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d8ea7-633f-41b1-99fc-5bf9b7c592ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:45.405261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d8ea7-633f-41b1-99fc-5bf9b7c592ab None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:45.405749 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d8ea7-633f-41b1-99fc-5bf9b7c592ab None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:45.406239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d8ea7-633f-41b1-99fc-5bf9b7c592ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:45.407375 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d8ea7-633f-41b1-99fc-5bf9b7c592ab None admin] RBAC: Adding query filter params (role.id=698ff821321a43d494fe8940cdd08564, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=b321586cdb3d4d25904472aaa4adba71) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:45.407999 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d8ea7-633f-41b1-99fc-5bf9b7c592ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:45.409669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d8ea7-633f-41b1-99fc-5bf9b7c592ab None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:45.416012 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 351/702] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:45.417295 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8ca7d-d88c-4f95-9583-16afbf1ecaf9 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:45.417729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 352/703] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:27:45 2022] GET /identity/v3/role_assignments?user.id=b321586cdb3d4d25904472aaa4adba71&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=698ff821321a43d494fe8940cdd08564 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:45.418924 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d8ea7-633f-41b1-99fc-5bf9b7c592ab None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:45.602838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8ca7d-d88c-4f95-9583-16afbf1ecaf9 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:45.622088 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:45.625351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d8ea7-633f-41b1-99fc-5bf9b7c592ab None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:45.649357 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c177b-5db7-4f96-b83f-3e8610056138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:45.650271 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c177b-5db7-4f96-b83f-3e8610056138 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:45.650381 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c177b-5db7-4f96-b83f-3e8610056138 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:45.650589 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c177b-5db7-4f96-b83f-3e8610056138 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:45.651316 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c177b-5db7-4f96-b83f-3e8610056138 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:45.651628 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c177b-5db7-4f96-b83f-3e8610056138 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:45.654186 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c177b-5db7-4f96-b83f-3e8610056138 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:45.668721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 352/704] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 23 22:27:45 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:45.670037 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c65099-39e2-4931-9291-d27ac866291a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:45.670256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6507334-b15a-4a29-b6d1-48f2af7085ea None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:45.670528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6507334-b15a-4a29-b6d1-48f2af7085ea None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:45.670817 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6507334-b15a-4a29-b6d1-48f2af7085ea None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:45.671645 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c65099-39e2-4931-9291-d27ac866291a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:45.672137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c65099-39e2-4931-9291-d27ac866291a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:45.672240 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 353/705] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:45.672734 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c65099-39e2-4931-9291-d27ac866291a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:45.682910 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa039a-7447-4a90-b908-35d05daf7b5e None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:45.682910 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa039a-7447-4a90-b908-35d05daf7b5e None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:45.682910 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa039a-7447-4a90-b908-35d05daf7b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:45.688262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:45.689603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c65099-39e2-4931-9291-d27ac866291a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:45.691803 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c65099-39e2-4931-9291-d27ac866291a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:45.709314 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-c3fa039a-7447-4a90-b908-35d05daf7b5e None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:45.711561 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c3fa039a-7447-4a90-b908-35d05daf7b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:45.713795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-06c65099-39e2-4931-9291-d27ac866291a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:45.713795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:45.713795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:45.713795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:45.713795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:45.713795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:45.713795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:45.713795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:45.713795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:45.713795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:45.713795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:45.713795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:45.713795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:45.713795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:45.713795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:45.714703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:45.714703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:45.714703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:45.714703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:45.714703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:45.714703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:45.714703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:45.714703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:45.714703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:45.714703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:45.714703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:45.714703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:45.714703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:45.714703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:45.714703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:45.714703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:45.714703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:45.715655 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:45.715655 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:45.715655 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:45.715655 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:45.715655 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 353/706] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:27:45.721214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c65099-39e2-4931-9291-d27ac866291a None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:45.819757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 354/707] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:45.853333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c65099-39e2-4931-9291-d27ac866291a None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:45.871727 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa039a-7447-4a90-b908-35d05daf7b5e None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:45.905385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1749e6-6845-4895-882f-af91348c8919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:45.906065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1749e6-6845-4895-882f-af91348c8919 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:45.906352 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1749e6-6845-4895-882f-af91348c8919 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:45.906632 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1749e6-6845-4895-882f-af91348c8919 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:45.907370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1749e6-6845-4895-882f-af91348c8919 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:45.907682 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1749e6-6845-4895-882f-af91348c8919 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:45.909104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1749e6-6845-4895-882f-af91348c8919 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:45.928989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 354/708] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:45.929961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1749e6-6845-4895-882f-af91348c8919 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:45.995089 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:46.000598 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa039a-7447-4a90-b908-35d05daf7b5e None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:46.015581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:46.048429 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1749e6-6845-4895-882f-af91348c8919 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:46.050200 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c13adf-dd16-4708-879e-b27ebfb8ada1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:46.052140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c13adf-dd16-4708-879e-b27ebfb8ada1 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:46.052443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c13adf-dd16-4708-879e-b27ebfb8ada1 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:46.052764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c13adf-dd16-4708-879e-b27ebfb8ada1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:46.069101 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c13adf-dd16-4708-879e-b27ebfb8ada1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:46.070639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c13adf-dd16-4708-879e-b27ebfb8ada1 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:46.087053 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047e2fc-d7f9-411f-9b5f-65124ca384ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:46.087934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047e2fc-d7f9-411f-9b5f-65124ca384ae None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:46.088292 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047e2fc-d7f9-411f-9b5f-65124ca384ae None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:46.088676 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047e2fc-d7f9-411f-9b5f-65124ca384ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:46.093781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-81c13adf-dd16-4708-879e-b27ebfb8ada1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:46.093781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:46.093781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:46.093781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:46.093781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:46.093781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:46.093781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:46.093781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:46.093781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:46.093781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:46.093781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:46.093781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:46.093781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:46.093781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:46.093781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:46.094884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:46.094884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:46.094884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:46.094884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:46.094884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:46.094884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:46.094884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:46.094884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:46.094884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:46.094884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:46.094884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:46.094884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:46.094884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:46.094884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:46.094884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:46.094884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:46.094884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:46.095834 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:46.095834 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:46.095834 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:46.095834 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:46.095834 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 355/709] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:27:46.096696 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1047e2fc-d7f9-411f-9b5f-65124ca384ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:46.098188 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1047e2fc-d7f9-411f-9b5f-65124ca384ae None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:46.108676 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c13adf-dd16-4708-879e-b27ebfb8ada1 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:46.121446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-1047e2fc-d7f9-411f-9b5f-65124ca384ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:46.121446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:46.121446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:46.121446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:46.121446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:46.121446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:46.121446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:46.121446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:46.121446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:46.121446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:46.121446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:46.121446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:46.121446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:46.121446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:46.121446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:46.122413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:46.122413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:46.122413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:46.122413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:46.122413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:46.122413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:46.122413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:46.122413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:46.122413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:46.122413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:46.122413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:46.122413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:46.122413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:46.122413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:46.122413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:46.122413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:46.122413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:46.123345 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:46.123345 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:46.123345 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:46.123345 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:46.123345 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:46.123345 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:46.123345 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:46.123345 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:46.128644 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 355/710] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:46.136516 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047e2fc-d7f9-411f-9b5f-65124ca384ae None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:46.250267 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047e2fc-d7f9-411f-9b5f-65124ca384ae None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:46.264471 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:46.286942 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f318d-1cb9-424c-9429-ecddd2ec11c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:46.288312 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f318d-1cb9-424c-9429-ecddd2ec11c4 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:46.288625 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f318d-1cb9-424c-9429-ecddd2ec11c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:46.288914 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f318d-1cb9-424c-9429-ecddd2ec11c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:46.289769 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f318d-1cb9-424c-9429-ecddd2ec11c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:46.290082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f318d-1cb9-424c-9429-ecddd2ec11c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:46.295237 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f318d-1cb9-424c-9429-ecddd2ec11c4 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:46.305351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 356/711] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:46.316718 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f318d-1cb9-424c-9429-ecddd2ec11c4 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:46.322193 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c13adf-dd16-4708-879e-b27ebfb8ada1 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:46.360744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2640a-eadf-4399-b13b-5323e4048504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:46.361914 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2640a-eadf-4399-b13b-5323e4048504 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:46.362524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2640a-eadf-4399-b13b-5323e4048504 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:46.363011 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2640a-eadf-4399-b13b-5323e4048504 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:46.364120 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c2640a-eadf-4399-b13b-5323e4048504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:46.364666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c2640a-eadf-4399-b13b-5323e4048504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:46.366604 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c2640a-eadf-4399-b13b-5323e4048504 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:46.379842 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 356/712] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:46.388600 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2640a-eadf-4399-b13b-5323e4048504 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:46.432090 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f318d-1cb9-424c-9429-ecddd2ec11c4 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:46.456113 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:46.483652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b91a1e-b0ce-4a0c-ba93-8ba8027ad6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:46.484317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b91a1e-b0ce-4a0c-ba93-8ba8027ad6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:46.484631 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b91a1e-b0ce-4a0c-ba93-8ba8027ad6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:46.484902 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b91a1e-b0ce-4a0c-ba93-8ba8027ad6c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:46.485741 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b91a1e-b0ce-4a0c-ba93-8ba8027ad6c2 None admin] RBAC: Adding query filter params (role.id=6ff7708b59984b9287cc81d46a57b4b4, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=159cc8befc3a4d57bbd313832084db62) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:46.486342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b91a1e-b0ce-4a0c-ba93-8ba8027ad6c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:46.487593 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b91a1e-b0ce-4a0c-ba93-8ba8027ad6c2 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:46.494314 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 357/713] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:27:46 2022] GET /identity/v3/role_assignments?user.id=159cc8befc3a4d57bbd313832084db62&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=6ff7708b59984b9287cc81d46a57b4b4 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:46.495540 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4990bf-e86d-4b9d-b7b9-53fffe882fbc None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:46.495859 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4990bf-e86d-4b9d-b7b9-53fffe882fbc None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:46.498929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4990bf-e86d-4b9d-b7b9-53fffe882fbc None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:46.498929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 358/714] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:46.503927 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681ab80-0663-48c5-9438-956834e412bc None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:46.504233 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681ab80-0663-48c5-9438-956834e412bc None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:46.504496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681ab80-0663-48c5-9438-956834e412bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:46.530099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-6681ab80-0663-48c5-9438-956834e412bc None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:46.532269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6681ab80-0663-48c5-9438-956834e412bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:46.556919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2640a-eadf-4399-b13b-5323e4048504 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:46.575737 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:46.595385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07134fc3-bc3d-4481-b88b-e12577a98a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:46.596130 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07134fc3-bc3d-4481-b88b-e12577a98a2e None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:46.596422 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07134fc3-bc3d-4481-b88b-e12577a98a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:46.596740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07134fc3-bc3d-4481-b88b-e12577a98a2e None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:46.597478 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07134fc3-bc3d-4481-b88b-e12577a98a2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:46.598974 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07134fc3-bc3d-4481-b88b-e12577a98a2e None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:46.630948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 359/715] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:27:46.655897 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681ab80-0663-48c5-9438-956834e412bc None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:46.708399 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 357/716] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 22:27:46 2022] POST /identity/v3/users => generated 254 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:27:46.774879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681ab80-0663-48c5-9438-956834e412bc None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:46.815251 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b8ac7-2c11-4dc5-8d38-977bc155839f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:46.816373 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b8ac7-2c11-4dc5-8d38-977bc155839f None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:46.816867 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b8ac7-2c11-4dc5-8d38-977bc155839f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:46.817428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b8ac7-2c11-4dc5-8d38-977bc155839f None admin] PATH_INFO: `/v3/services/network` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:46.818322 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b8ac7-2c11-4dc5-8d38-977bc155839f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:46.821120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b8ac7-2c11-4dc5-8d38-977bc155839f None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:46.838129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-ca2b8ac7-2c11-4dc5-8d38-977bc155839f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 22:27:46.838129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:46.838129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:27:46.838129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:27:46.838129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:27:46.838129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:27:46.838129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:27:46.838129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:27:46.838129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 22:27:46.838129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:46.838129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:27:46.838129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:46.838129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:46.838129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:46.838129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:46.838129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:46.838129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:46.839186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:46.839186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:46.839186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:46.839186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:46.839186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:46.839186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:46.839186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:27:46.839186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:27:46.839186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:27:46.839186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:27:46.839186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:46.839186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:46.839186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:46.839186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:46.839186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:46.839186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:46.840023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:46.840023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:46.840023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:46.840023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:46.840023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:46.840023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:46.840023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:46.840023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:46.840023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:46.840023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:46.840023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:27:46.840023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:27:46.840023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 22:27:46.840023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:46.842556 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 360/717] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:46 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:27:46.848634 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07134fc3-bc3d-4481-b88b-e12577a98a2e None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:46.956095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:46.982005 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07134fc3-bc3d-4481-b88b-e12577a98a2e None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:47.038964 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17a22f-f1c7-4b33-bd18-1f5b10d26479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:47.039979 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a22f-f1c7-4b33-bd18-1f5b10d26479 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:47.040434 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a22f-f1c7-4b33-bd18-1f5b10d26479 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:47.040903 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a22f-f1c7-4b33-bd18-1f5b10d26479 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:47.042114 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17a22f-f1c7-4b33-bd18-1f5b10d26479 None admin] RBAC: Adding query filter params (name=network) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:47.042612 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17a22f-f1c7-4b33-bd18-1f5b10d26479 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:47.044229 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17a22f-f1c7-4b33-bd18-1f5b10d26479 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:47.052939 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 358/718] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 22:27:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:47.056280 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b8ac7-2c11-4dc5-8d38-977bc155839f None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:47.092895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:47.169562 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b8ac7-2c11-4dc5-8d38-977bc155839f None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:47.211402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b280ae-8a03-45b1-8876-e35d0bf934a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:47.212381 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b280ae-8a03-45b1-8876-e35d0bf934a4 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:47.212869 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b280ae-8a03-45b1-8876-e35d0bf934a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:47.213329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b280ae-8a03-45b1-8876-e35d0bf934a4 None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:47.214253 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b280ae-8a03-45b1-8876-e35d0bf934a4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:47.214741 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b280ae-8a03-45b1-8876-e35d0bf934a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:47.216337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b280ae-8a03-45b1-8876-e35d0bf934a4 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:47.222916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 361/719] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 22:27:47 2022] GET /identity/v3/services?type=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:48.203767 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0f77c-b8bb-40de-b0e1-63da38a48b82 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:48.204839 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0f77c-b8bb-40de-b0e1-63da38a48b82 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:48.205337 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0f77c-b8bb-40de-b0e1-63da38a48b82 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:48.207168 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 359/720] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:48.213002 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc45367-8204-4576-8f28-92c2a1387572 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:48.213856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc45367-8204-4576-8f28-92c2a1387572 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:48.214341 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc45367-8204-4576-8f28-92c2a1387572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:48.235735 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-6cc45367-8204-4576-8f28-92c2a1387572 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:48.237274 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6cc45367-8204-4576-8f28-92c2a1387572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:48.310867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8795110-3f32-41fd-b13f-35c69c608f20 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:48.311183 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8795110-3f32-41fd-b13f-35c69c608f20 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:48.311597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8795110-3f32-41fd-b13f-35c69c608f20 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:48.313416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 360/721] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:48.319664 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17363d83-b6da-4366-b856-3e003448ab03 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:48.320268 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17363d83-b6da-4366-b856-3e003448ab03 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:48.320561 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17363d83-b6da-4366-b856-3e003448ab03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:48.340709 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-17363d83-b6da-4366-b856-3e003448ab03 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:48.342071 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-17363d83-b6da-4366-b856-3e003448ab03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:48.355914 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 362/722] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:48.395106 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc45367-8204-4576-8f28-92c2a1387572 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:48.415975 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 361/723] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:27:48.439912 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17363d83-b6da-4366-b856-3e003448ab03 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:48.464196 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:48.526157 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:48.551812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17363d83-b6da-4366-b856-3e003448ab03 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:48.589354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca6050-402b-4cbd-bfb1-7e186e33343c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:48.590037 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca6050-402b-4cbd-bfb1-7e186e33343c None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:48.590322 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca6050-402b-4cbd-bfb1-7e186e33343c None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:48.590602 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca6050-402b-4cbd-bfb1-7e186e33343c None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:48.591343 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca6050-402b-4cbd-bfb1-7e186e33343c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:48.592757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca6050-402b-4cbd-bfb1-7e186e33343c None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:48.630059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc45367-8204-4576-8f28-92c2a1387572 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:48.665744 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f55ea-0682-4379-8f63-3aebc3a5c69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:48.666841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f55ea-0682-4379-8f63-3aebc3a5c69b None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:48.667132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f55ea-0682-4379-8f63-3aebc3a5c69b None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:48.667417 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f55ea-0682-4379-8f63-3aebc3a5c69b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:48.678714 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f55ea-0682-4379-8f63-3aebc3a5c69b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:48.686545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-e83f55ea-0682-4379-8f63-3aebc3a5c69b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:27:48.686545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:48.686545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:48.686545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:48.686545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:48.686545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:48.686545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:48.686545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:48.686545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:48.686545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:48.686545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:48.686545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:48.686545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:48.686545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:48.686545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:48.686545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:48.687535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:48.687535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:48.687535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:48.687535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:48.687535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:48.687535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:48.687535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:48.687535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:48.687535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:48.687535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:48.687535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:48.687535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:48.687535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:48.687535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:48.687535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:48.687535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:48.688368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:48.688368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:48.688368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:48.688368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:48.688368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:48.688368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:48.688368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:48.688368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:48.688368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:48.688368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:48.688368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:27:48.688368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:48.688368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 363/724] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:27:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:48.696058 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f55ea-0682-4379-8f63-3aebc3a5c69b None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:48.724458 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 362/725] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 22:27:48 2022] POST /identity/v3/services => generated 251 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:27:48.792551 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:48.810308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f55ea-0682-4379-8f63-3aebc3a5c69b None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:48.866787 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9b58f-c9b9-429a-9a13-848cfa3a3b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:48.867495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9b58f-c9b9-429a-9a13-848cfa3a3b3f None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:48.867788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9b58f-c9b9-429a-9a13-848cfa3a3b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:48.868070 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9b58f-c9b9-429a-9a13-848cfa3a3b3f None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:48.868876 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9b58f-c9b9-429a-9a13-848cfa3a3b3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:48.869192 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9b58f-c9b9-429a-9a13-848cfa3a3b3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:48.870619 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9b58f-c9b9-429a-9a13-848cfa3a3b3f None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:48.884742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 364/726] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:48.889736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca6050-402b-4cbd-bfb1-7e186e33343c None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:49.020391 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca6050-402b-4cbd-bfb1-7e186e33343c None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:49.050572 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:49.055587 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb8707-4ac1-49bb-a082-0f1b229a0866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:49.056393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb8707-4ac1-49bb-a082-0f1b229a0866 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:49.056712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb8707-4ac1-49bb-a082-0f1b229a0866 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:49.056988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb8707-4ac1-49bb-a082-0f1b229a0866 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:49.067243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb8707-4ac1-49bb-a082-0f1b229a0866 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:49.069257 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb8707-4ac1-49bb-a082-0f1b229a0866 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:49.088250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-cfbb8707-4ac1-49bb-a082-0f1b229a0866 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:49.088250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:49.088250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:49.088250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:49.088250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:49.088250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:49.088250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:49.088250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:49.088250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:49.088250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:49.088250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:49.088250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:49.088250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:49.088250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:49.088250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:49.089209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:49.089209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:49.089209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:49.089209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:49.089209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:49.089209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:49.089209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:49.089209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:49.089209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:49.089209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:49.089209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:49.089209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:49.089209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:49.089209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:49.089209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:49.089209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:49.089209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:49.090100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:49.090100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:49.090100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:49.090100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:49.090100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 363/727] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:49.094985 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9b58f-c9b9-429a-9a13-848cfa3a3b3f None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:49.217945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9b58f-c9b9-429a-9a13-848cfa3a3b3f None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:49.254174 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72812612-652e-40e5-a7a7-dda45455028a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:49.254849 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72812612-652e-40e5-a7a7-dda45455028a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:49.255139 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72812612-652e-40e5-a7a7-dda45455028a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:49.255421 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72812612-652e-40e5-a7a7-dda45455028a None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:49.256743 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72812612-652e-40e5-a7a7-dda45455028a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:49.257058 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72812612-652e-40e5-a7a7-dda45455028a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:49.258447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72812612-652e-40e5-a7a7-dda45455028a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:49.276678 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 365/728] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:49.281618 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb8707-4ac1-49bb-a082-0f1b229a0866 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:49.356639 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7275e61-66a3-44b8-ae2e-9cb5389921d4 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:49.356985 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7275e61-66a3-44b8-ae2e-9cb5389921d4 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:49.357212 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7275e61-66a3-44b8-ae2e-9cb5389921d4 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:49.358855 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 366/729] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:49.364057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6f0a1-f91d-45fa-9e9f-cd0714fbffda None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:49.368710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6f0a1-f91d-45fa-9e9f-cd0714fbffda None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:49.368710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6f0a1-f91d-45fa-9e9f-cd0714fbffda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:49.383053 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:49.390481 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-c7d6f0a1-f91d-45fa-9e9f-cd0714fbffda None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:49.391706 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-c7d6f0a1-f91d-45fa-9e9f-cd0714fbffda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:49.456869 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb8707-4ac1-49bb-a082-0f1b229a0866 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:49.524100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a632b3-babb-4ebd-aaa9-d8443b61232e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:49.524936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a632b3-babb-4ebd-aaa9-d8443b61232e None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:49.525218 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a632b3-babb-4ebd-aaa9-d8443b61232e None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:49.525494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a632b3-babb-4ebd-aaa9-d8443b61232e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:49.528636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 367/730] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:49 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:49.535784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a632b3-babb-4ebd-aaa9-d8443b61232e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:49.537830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a632b3-babb-4ebd-aaa9-d8443b61232e None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:49.553848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-d2a632b3-babb-4ebd-aaa9-d8443b61232e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:27:49.553848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:49.553848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:49.553848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:49.553848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:49.553848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:49.553848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:49.553848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:49.553848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:49.553848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:49.553848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:49.553848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:49.553848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:49.553848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:49.553848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:49.553848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:49.554835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:49.554835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:49.554835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:49.554835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:49.554835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:49.554835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:49.554835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:49.554835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:49.554835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:49.554835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:49.554835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:49.554835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:49.554835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:49.554835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:49.554835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:49.554835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:49.554835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:49.554835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:49.555776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:49.555776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:49.555776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:49.555776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:49.555776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:49.555776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:49.555776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:49.555776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:49.555776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:27:49.555776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:49.555776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 364/731] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:27:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:27:49.560845 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d6f0a1-f91d-45fa-9e9f-cd0714fbffda None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:49.578187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca540af-c25b-4fdf-bc71-d93ab157345b None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:49.578479 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca540af-c25b-4fdf-bc71-d93ab157345b None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:49.578742 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca540af-c25b-4fdf-bc71-d93ab157345b None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:49.580367 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 365/732] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:27:49.581107 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca540af-c25b-4fdf-bc71-d93ab157345b None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:49.633432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:49.677326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d6f0a1-f91d-45fa-9e9f-cd0714fbffda None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:49.692355 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca540af-c25b-4fdf-bc71-d93ab157345b None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:49.711055 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:49.716512 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa3c34-9177-41fc-bf24-c0b82c002d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:49.717458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa3c34-9177-41fc-bf24-c0b82c002d22 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:49.717809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa3c34-9177-41fc-bf24-c0b82c002d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:49.718146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa3c34-9177-41fc-bf24-c0b82c002d22 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:49.719074 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa3c34-9177-41fc-bf24-c0b82c002d22 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:49.719448 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa3c34-9177-41fc-bf24-c0b82c002d22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:49.721070 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa3c34-9177-41fc-bf24-c0b82c002d22 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:49.733386 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1c517-890e-48ed-aab2-a2089b4e4189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:49.734497 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1c517-890e-48ed-aab2-a2089b4e4189 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:49.734926 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1c517-890e-48ed-aab2-a2089b4e4189 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:49.735368 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1c517-890e-48ed-aab2-a2089b4e4189 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:49.746256 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 368/733] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 23 22:27:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:49.746256 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c566c14-2dce-4fdc-aedd-280fbe837558 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:49.746256 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c566c14-2dce-4fdc-aedd-280fbe837558 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:49.746256 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c566c14-2dce-4fdc-aedd-280fbe837558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:49.753432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1c517-890e-48ed-aab2-a2089b4e4189 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:49.762185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-f3d1c517-890e-48ed-aab2-a2089b4e4189 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:27:49.762185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:49.762185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:49.762185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:49.762185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:49.762185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:49.762185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:49.762185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:49.762185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:49.762185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:49.762185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:49.762185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:49.762185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:49.762185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:49.762185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:49.762185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:49.763162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:49.763162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:49.763162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:49.763162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:49.763162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:49.763162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:49.763162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:49.763162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:49.763162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:49.763162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:49.763162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:49.763162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:49.763162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:49.763162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:49.763162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:49.763162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:49.764048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:49.764048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:49.764048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:49.764048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:49.764048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:49.764048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:49.764048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:49.764048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:49.764048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:49.764048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:49.764048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:27:49.764048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:49.764719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-8c566c14-2dce-4fdc-aedd-280fbe837558 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:49.764719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-8c566c14-2dce-4fdc-aedd-280fbe837558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:49.766526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 366/734] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:27:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:49.767602 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1c517-890e-48ed-aab2-a2089b4e4189 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:49.840052 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 369/735] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:49 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:49.841129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c566c14-2dce-4fdc-aedd-280fbe837558 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:49.880779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1c517-890e-48ed-aab2-a2089b4e4189 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:49.884061 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:49.917789 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb783e24-886e-432a-9fe0-1596147c5b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:49.918437 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb783e24-886e-432a-9fe0-1596147c5b48 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:49.918722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb783e24-886e-432a-9fe0-1596147c5b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:49.918999 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb783e24-886e-432a-9fe0-1596147c5b48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:49.953580 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c566c14-2dce-4fdc-aedd-280fbe837558 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:49.975255 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:49.991369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629f403-7390-4158-89d2-7c859f6b852b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:49.992011 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629f403-7390-4158-89d2-7c859f6b852b None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:49.992297 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629f403-7390-4158-89d2-7c859f6b852b None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:49.992600 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629f403-7390-4158-89d2-7c859f6b852b None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:49.993342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629f403-7390-4158-89d2-7c859f6b852b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:49.993590 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb783e24-886e-432a-9fe0-1596147c5b48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:49.993692 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629f403-7390-4158-89d2-7c859f6b852b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:49.994999 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629f403-7390-4158-89d2-7c859f6b852b None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:49.995095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb783e24-886e-432a-9fe0-1596147c5b48 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:50.008612 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 370/736] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:50.008612 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629f403-7390-4158-89d2-7c859f6b852b None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:50.121975 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629f403-7390-4158-89d2-7c859f6b852b None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:50.159115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ada8ec-1582-4aff-aa9f-aa87475da0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:50.159918 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ada8ec-1582-4aff-aa9f-aa87475da0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:50.160315 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ada8ec-1582-4aff-aa9f-aa87475da0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:50.160611 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ada8ec-1582-4aff-aa9f-aa87475da0d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:50.172295 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ada8ec-1582-4aff-aa9f-aa87475da0d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:50.180054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-83ada8ec-1582-4aff-aa9f-aa87475da0d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:50.180054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:50.180054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:50.180054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:50.180054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:50.180054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:50.180054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:50.180054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:50.180054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:50.180054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:50.180054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:50.180054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:50.180054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:50.180054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:50.180054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:50.180054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:50.181346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:50.181346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:50.181346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:50.181346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:50.181346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:50.181346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:50.181346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:50.181346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:50.181346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:50.181346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:50.181346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:50.181346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:50.181346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:50.181346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:50.181346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:50.181346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:50.182386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:50.182386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:50.182386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:50.182386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:50.182386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:50.182386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:50.182386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:50.182386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:50.182386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:50.182386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:50.182386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:50.182386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:50.182386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 371/737] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:27:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:50.183258 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b06fea-497f-4371-afb5-a99882bef89a None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:50.183258 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b06fea-497f-4371-afb5-a99882bef89a None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:50.183444 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b06fea-497f-4371-afb5-a99882bef89a None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:50.184942 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 372/738] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:50.185549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b06fea-497f-4371-afb5-a99882bef89a None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:50.228615 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 367/739] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:49 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:27:50.229370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64662d76-d8bf-46f3-8f9b-a98a842939ec None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:50.229661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64662d76-d8bf-46f3-8f9b-a98a842939ec None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:50.229965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64662d76-d8bf-46f3-8f9b-a98a842939ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:50.235444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:50.242504 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:50.246256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-64662d76-d8bf-46f3-8f9b-a98a842939ec None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:50.247448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-64662d76-d8bf-46f3-8f9b-a98a842939ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:50.311879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b06fea-497f-4371-afb5-a99882bef89a None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:50.332597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 368/740] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:50 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:27:50.332597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64662d76-d8bf-46f3-8f9b-a98a842939ec None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:50.347817 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a370a8a-9242-4ced-9553-a44b1356dd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:50.348729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a370a8a-9242-4ced-9553-a44b1356dd4c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:50.349022 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a370a8a-9242-4ced-9553-a44b1356dd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:50.349308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a370a8a-9242-4ced-9553-a44b1356dd4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:50.362590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a370a8a-9242-4ced-9553-a44b1356dd4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:50.364031 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a370a8a-9242-4ced-9553-a44b1356dd4c None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:50.384068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-3a370a8a-9242-4ced-9553-a44b1356dd4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:50.384068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:50.384068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:50.384068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:50.384068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:50.384068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:50.384068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:50.384068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:50.384068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:50.384068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:50.384068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:50.384068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:50.384068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:50.384068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:50.384068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:50.385030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:50.385030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:50.385030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:50.385030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:50.385030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:50.385030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:50.385030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:50.385030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:50.385030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:50.385030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:50.385030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:50.385030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:50.385030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:50.385030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:50.385030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:50.385030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:50.385030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:50.385928 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:50.385928 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:50.385928 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:50.385928 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:50.385928 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 373/741] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:50.386219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a370a8a-9242-4ced-9553-a44b1356dd4c None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:50.445180 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64662d76-d8bf-46f3-8f9b-a98a842939ec None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:50.482282 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144133d5-b9c5-4a00-b89e-60efb0af8fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:50.483230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144133d5-b9c5-4a00-b89e-60efb0af8fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:50.483521 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144133d5-b9c5-4a00-b89e-60efb0af8fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:50.483949 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144133d5-b9c5-4a00-b89e-60efb0af8fa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:50.484082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:50.484740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144133d5-b9c5-4a00-b89e-60efb0af8fa5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:50.485057 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144133d5-b9c5-4a00-b89e-60efb0af8fa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:50.487053 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144133d5-b9c5-4a00-b89e-60efb0af8fa5 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:50.489053 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:50.491704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a370a8a-9242-4ced-9553-a44b1356dd4c None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:50.500718 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 369/742] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:27:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:50.501530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144133d5-b9c5-4a00-b89e-60efb0af8fa5 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:50.533769 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bc43f-6344-4f52-b4fc-daf28611bb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:50.534510 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bc43f-6344-4f52-b4fc-daf28611bb23 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:50.534802 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bc43f-6344-4f52-b4fc-daf28611bb23 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:50.535087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bc43f-6344-4f52-b4fc-daf28611bb23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:50.547008 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bc43f-6344-4f52-b4fc-daf28611bb23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:50.548462 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bc43f-6344-4f52-b4fc-daf28611bb23 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:50.566560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-7d7bc43f-6344-4f52-b4fc-daf28611bb23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:50.566560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:50.566560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:50.566560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:50.566560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:50.566560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:50.566560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:50.566560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:50.566560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:50.566560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:50.566560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:50.566560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:50.566560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:50.566560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:50.566560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:50.567483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:50.567483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:50.567483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:50.567483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:50.567483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:50.567483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:50.567483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:50.567483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:50.567483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:50.567483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:50.567483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:50.567483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:50.567483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:50.567483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:50.567483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:50.567483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:50.567483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:50.568433 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:50.568433 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:50.568433 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:50.568433 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:50.568433 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 374/743] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:50.569276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bc43f-6344-4f52-b4fc-daf28611bb23 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:50.613784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144133d5-b9c5-4a00-b89e-60efb0af8fa5 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:50.651891 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712f241-5e90-4030-b503-6f836ca0e443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:50.652479 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712f241-5e90-4030-b503-6f836ca0e443 None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:50.652793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712f241-5e90-4030-b503-6f836ca0e443 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:50.653073 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712f241-5e90-4030-b503-6f836ca0e443 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:50.653799 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712f241-5e90-4030-b503-6f836ca0e443 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:50.655167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712f241-5e90-4030-b503-6f836ca0e443 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:50.677664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bc43f-6344-4f52-b4fc-daf28611bb23 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:50.711117 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf66b1-e17a-4610-9901-234318a82265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:50.711784 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf66b1-e17a-4610-9901-234318a82265 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:50.712081 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf66b1-e17a-4610-9901-234318a82265 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:50.712365 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf66b1-e17a-4610-9901-234318a82265 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:50.713133 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cf66b1-e17a-4610-9901-234318a82265 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:50.713445 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cf66b1-e17a-4610-9901-234318a82265 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:50.714826 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cf66b1-e17a-4610-9901-234318a82265 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:50.720022 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 370/744] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 22:27:50 2022] POST /identity/v3/services => generated 240 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:27:50.733023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 375/745] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:50.733632 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712f241-5e90-4030-b503-6f836ca0e443 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:50.733857 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf66b1-e17a-4610-9901-234318a82265 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:50.744427 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:50.825066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:50.879660 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf66b1-e17a-4610-9901-234318a82265 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:50.900385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712f241-5e90-4030-b503-6f836ca0e443 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:50.929683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2465a-0475-4227-a823-d4c88312348e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:50.930820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2465a-0475-4227-a823-d4c88312348e None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:50.931170 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2465a-0475-4227-a823-d4c88312348e None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:50.931518 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2465a-0475-4227-a823-d4c88312348e None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:50.932326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2465a-0475-4227-a823-d4c88312348e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:50.932767 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2465a-0475-4227-a823-d4c88312348e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:50.934234 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2465a-0475-4227-a823-d4c88312348e None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:50.943143 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee746b2-ab9c-47e1-8558-fae1c0b7c2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:50.945599 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee746b2-ab9c-47e1-8558-fae1c0b7c2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:50.946084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee746b2-ab9c-47e1-8558-fae1c0b7c2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:50.946828 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee746b2-ab9c-47e1-8558-fae1c0b7c2c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:50.948923 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 376/746] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:50.949831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2465a-0475-4227-a823-d4c88312348e None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:50.959746 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee746b2-ab9c-47e1-8558-fae1c0b7c2c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:50.961499 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee746b2-ab9c-47e1-8558-fae1c0b7c2c9 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:50.985606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-8ee746b2-ab9c-47e1-8558-fae1c0b7c2c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:50.985606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:50.985606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:50.985606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:50.985606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:50.985606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:50.985606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:50.985606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:50.985606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:50.985606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:50.985606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:50.985606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:50.985606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:50.985606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:50.985606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:50.986549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:50.986549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:50.986549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:50.986549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:50.986549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:50.986549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:50.986549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:50.986549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:50.986549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:50.986549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:50.986549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:50.986549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:50.986549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:50.986549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:50.986549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:50.986549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:50.986549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:50.987421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:50.987421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:50.987421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:50.987421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:50.990033 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 371/747] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:50.992186 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee746b2-ab9c-47e1-8558-fae1c0b7c2c9 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:51.024387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:51.059633 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2465a-0475-4227-a823-d4c88312348e None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:51.097099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d1636-93f9-4c43-aa33-3b254e88711e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:51.098013 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d1636-93f9-4c43-aa33-3b254e88711e None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:51.098309 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d1636-93f9-4c43-aa33-3b254e88711e None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:51.098585 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d1636-93f9-4c43-aa33-3b254e88711e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:51.109057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d1636-93f9-4c43-aa33-3b254e88711e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:51.110834 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d1636-93f9-4c43-aa33-3b254e88711e None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:51.119421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee746b2-ab9c-47e1-8558-fae1c0b7c2c9 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:51.121850 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:51.130271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-204d1636-93f9-4c43-aa33-3b254e88711e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:27:51.130271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:51.130271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:51.130271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:51.130271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:51.130271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:51.130271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:51.130271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:51.130271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:51.130271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:51.130271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:51.130271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:51.130271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:51.130271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:51.130271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:51.130271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:51.135184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:51.135184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:51.135184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:51.135184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:51.135184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:51.135184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:51.135184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:51.135184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:51.135184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:51.135184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:51.135184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:51.135184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:51.135184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:51.135184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:51.135184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:51.135184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:51.135184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:51.135184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:51.136337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:51.136337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:51.136337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:51.136337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:51.136337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:51.136337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:51.136337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:51.136337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:51.136337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:27:51.136337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:51.136337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 377/748] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 23 22:27:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:51.136337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d1636-93f9-4c43-aa33-3b254e88711e None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:51.159442 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f159f2-2992-4d47-a5c1-5253b04354af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:51.160868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f159f2-2992-4d47-a5c1-5253b04354af None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:51.161082 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f159f2-2992-4d47-a5c1-5253b04354af None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:51.161366 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f159f2-2992-4d47-a5c1-5253b04354af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:51.169445 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f159f2-2992-4d47-a5c1-5253b04354af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:51.170898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f159f2-2992-4d47-a5c1-5253b04354af None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:51.185108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-b3f159f2-2992-4d47-a5c1-5253b04354af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:51.185108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:51.185108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:51.185108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:51.185108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:51.185108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:51.185108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:51.185108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:51.185108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:51.185108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:51.185108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:51.185108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:51.185108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:51.185108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:51.185108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:51.186162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:51.186162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:51.186162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:51.186162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:51.186162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:51.186162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:51.186162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:51.186162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:51.186162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:51.186162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:51.186162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:51.186162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:51.186162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:51.186162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:51.186162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:51.186162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:51.186162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:51.187054 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:51.187054 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:51.187054 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:51.187054 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:51.187054 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:51.187054 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:51.187054 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:51.187054 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:51.187054 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 372/749] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:51.187054 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f159f2-2992-4d47-a5c1-5253b04354af None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:51.254175 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d1636-93f9-4c43-aa33-3b254e88711e None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:51.281577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:51.291775 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf46d8-0913-4cbd-8ca8-80dfb043b581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:51.292423 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf46d8-0913-4cbd-8ca8-80dfb043b581 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:51.292746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf46d8-0913-4cbd-8ca8-80dfb043b581 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:51.293028 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf46d8-0913-4cbd-8ca8-80dfb043b581 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:51.294349 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf46d8-0913-4cbd-8ca8-80dfb043b581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:51.294349 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf46d8-0913-4cbd-8ca8-80dfb043b581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:51.295424 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf46d8-0913-4cbd-8ca8-80dfb043b581 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:51.311454 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 378/750] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:51.311454 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d15d24-627a-48f5-9d84-e2021e7aefcd None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:51.311454 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d15d24-627a-48f5-9d84-e2021e7aefcd None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:51.311454 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d15d24-627a-48f5-9d84-e2021e7aefcd None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:51.311454 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 379/751] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:51.311454 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d15d24-627a-48f5-9d84-e2021e7aefcd None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:51.314933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f159f2-2992-4d47-a5c1-5253b04354af None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:51.351282 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24070f-79ed-440a-8deb-4e6cae42ee49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:51.351955 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24070f-79ed-440a-8deb-4e6cae42ee49 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:51.352242 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24070f-79ed-440a-8deb-4e6cae42ee49 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:51.352518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24070f-79ed-440a-8deb-4e6cae42ee49 None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:51.353414 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24070f-79ed-440a-8deb-4e6cae42ee49 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:51.353686 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24070f-79ed-440a-8deb-4e6cae42ee49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:51.355114 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24070f-79ed-440a-8deb-4e6cae42ee49 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:51.389869 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 373/752] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 23 22:27:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:51.390384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24070f-79ed-440a-8deb-4e6cae42ee49 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:51.415502 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:51.428436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d15d24-627a-48f5-9d84-e2021e7aefcd None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:51.465210 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af6ac4-908b-4a43-ad3a-1d6e36cbc6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:51.465818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af6ac4-908b-4a43-ad3a-1d6e36cbc6db None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:51.466105 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af6ac4-908b-4a43-ad3a-1d6e36cbc6db None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:51.466382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af6ac4-908b-4a43-ad3a-1d6e36cbc6db None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:51.467175 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af6ac4-908b-4a43-ad3a-1d6e36cbc6db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:51.467484 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af6ac4-908b-4a43-ad3a-1d6e36cbc6db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:51.468940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af6ac4-908b-4a43-ad3a-1d6e36cbc6db None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:51.485810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 380/753] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:51.485810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1026ab-1bd4-493d-ad3b-02cf2eb25eb3 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:51.485810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1026ab-1bd4-493d-ad3b-02cf2eb25eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:51.485810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1026ab-1bd4-493d-ad3b-02cf2eb25eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:51.499680 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-3c1026ab-1bd4-493d-ad3b-02cf2eb25eb3 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:51.500901 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3c1026ab-1bd4-493d-ad3b-02cf2eb25eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:51.502555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24070f-79ed-440a-8deb-4e6cae42ee49 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:51.538140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea3d68-68f1-4759-9b65-f570e27a2b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:51.538963 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea3d68-68f1-4759-9b65-f570e27a2b02 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:51.539252 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea3d68-68f1-4759-9b65-f570e27a2b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:51.539532 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea3d68-68f1-4759-9b65-f570e27a2b02 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:51.546822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:51.547075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea3d68-68f1-4759-9b65-f570e27a2b02 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:51.548658 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea3d68-68f1-4759-9b65-f570e27a2b02 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:51.565431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-57ea3d68-68f1-4759-9b65-f570e27a2b02 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:27:51.565431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:51.565431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:51.565431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:51.565431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:51.565431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:51.565431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:51.565431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:51.565431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:51.565431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:51.565431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:51.565431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:51.565431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:51.565431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:51.565431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:51.565431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:51.566410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:51.566410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:51.566410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:51.566410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:51.566410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:51.566410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:51.566410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:51.566410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:51.566410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:51.566410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:51.566410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:51.566410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:51.566410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:51.566410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:51.566410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:51.566410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:51.566410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:51.566410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:51.567374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:51.567374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:51.567374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:51.567374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:51.567374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:51.567374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:51.567374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:51.567374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:51.567374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:27:51.567374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:51.567374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 374/754] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 23 22:27:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:51.571896 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea3d68-68f1-4759-9b65-f570e27a2b02 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:51.580632 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 381/755] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:27:51.580632 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1026ab-1bd4-493d-ad3b-02cf2eb25eb3 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:51.676536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:51.681461 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1026ab-1bd4-493d-ad3b-02cf2eb25eb3 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:51.694734 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea3d68-68f1-4759-9b65-f570e27a2b02 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:51.717102 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60728d-e850-48ac-afcb-a00fd77bf886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:51.717690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60728d-e850-48ac-afcb-a00fd77bf886 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:51.717972 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60728d-e850-48ac-afcb-a00fd77bf886 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:51.718250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60728d-e850-48ac-afcb-a00fd77bf886 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:51.719074 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60728d-e850-48ac-afcb-a00fd77bf886 None admin] RBAC: Adding query filter params (role.id=6ff7708b59984b9287cc81d46a57b4b4, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=b321586cdb3d4d25904472aaa4adba71) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:51.719383 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60728d-e850-48ac-afcb-a00fd77bf886 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:51.720823 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60728d-e850-48ac-afcb-a00fd77bf886 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:51.736612 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 382/756] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:27:51 2022] GET /identity/v3/role_assignments?user.id=b321586cdb3d4d25904472aaa4adba71&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=6ff7708b59984b9287cc81d46a57b4b4 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:51.736612 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60728d-e850-48ac-afcb-a00fd77bf886 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:51.744735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e4eb6-890e-43d6-b737-8da80be412a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:51.745676 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e4eb6-890e-43d6-b737-8da80be412a7 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:51.746031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e4eb6-890e-43d6-b737-8da80be412a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:51.746375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e4eb6-890e-43d6-b737-8da80be412a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:51.758443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e4eb6-890e-43d6-b737-8da80be412a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:51.760031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e4eb6-890e-43d6-b737-8da80be412a7 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:51.779909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-768e4eb6-890e-43d6-b737-8da80be412a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:51.779909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:51.779909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:51.779909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:51.779909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:51.779909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:51.779909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:51.779909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:51.779909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:51.779909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:51.779909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:51.779909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:51.779909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:51.779909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:51.779909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:51.780908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:51.780908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:51.780908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:51.780908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:51.780908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:51.780908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:51.780908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:51.780908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:51.780908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:51.780908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:51.780908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:51.780908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:51.780908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:51.780908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:51.780908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:51.780908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:51.780908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:51.781920 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:51.781920 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:51.781920 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:51.781920 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:51.781920 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 375/757] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:51.782738 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e4eb6-890e-43d6-b737-8da80be412a7 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:51.789393 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:51.857492 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60728d-e850-48ac-afcb-a00fd77bf886 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:51.895861 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e4eb6-890e-43d6-b737-8da80be412a7 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:51.903862 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83a1a2-1d02-4cba-a9f9-a40c7b4682b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:51.904561 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83a1a2-1d02-4cba-a9f9-a40c7b4682b8 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:51.904893 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83a1a2-1d02-4cba-a9f9-a40c7b4682b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:51.905177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83a1a2-1d02-4cba-a9f9-a40c7b4682b8 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:51.906039 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83a1a2-1d02-4cba-a9f9-a40c7b4682b8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:51.906357 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83a1a2-1d02-4cba-a9f9-a40c7b4682b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:51.907800 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83a1a2-1d02-4cba-a9f9-a40c7b4682b8 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:51.935327 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 383/758] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 23 22:27:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:51.935327 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83a1a2-1d02-4cba-a9f9-a40c7b4682b8 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:51.949396 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd972ad-71e4-4866-ada9-37b4a479246a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:51.950654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd972ad-71e4-4866-ada9-37b4a479246a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:51.951176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd972ad-71e4-4866-ada9-37b4a479246a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:51.951661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd972ad-71e4-4866-ada9-37b4a479246a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:51.952511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd972ad-71e4-4866-ada9-37b4a479246a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:51.954289 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd972ad-71e4-4866-ada9-37b4a479246a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:51.956456 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:51.981432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-bdd972ad-71e4-4866-ada9-37b4a479246a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 22:27:51.981432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:51.981432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:27:51.981432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:27:51.981432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:27:51.981432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:27:51.981432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:27:51.981432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:27:51.981432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 22:27:51.981432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:51.981432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:27:51.981432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:51.981432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:51.981432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:51.981432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:51.981432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:51.982431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:51.982431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:51.982431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:51.982431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:51.982431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:51.982431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:51.982431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:51.982431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:27:51.982431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:27:51.982431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:27:51.982431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:27:51.982431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:51.982431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:51.982431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:51.982431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:51.982431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:51.983293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:51.983293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:51.983293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:51.983293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:51.983293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:51.983293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:51.983293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:51.983293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:51.983293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:51.983293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:51.983293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:51.983293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:27:51.983293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:27:51.983293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 22:27:51.983293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:51.983293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 376/759] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Jun 23 22:27:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:51.984153 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd972ad-71e4-4866-ada9-37b4a479246a None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:52.041463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:52.045282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83a1a2-1d02-4cba-a9f9-a40c7b4682b8 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:52.083164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c0cb3-f4f9-4aea-a39f-063119652f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:52.083836 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c0cb3-f4f9-4aea-a39f-063119652f9c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:52.084314 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c0cb3-f4f9-4aea-a39f-063119652f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:52.084624 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c0cb3-f4f9-4aea-a39f-063119652f9c None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:52.085478 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c0cb3-f4f9-4aea-a39f-063119652f9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:52.085794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c0cb3-f4f9-4aea-a39f-063119652f9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:52.087284 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c0cb3-f4f9-4aea-a39f-063119652f9c None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:52.092251 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd972ad-71e4-4866-ada9-37b4a479246a None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:52.104688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 384/760] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:52.104688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c0cb3-f4f9-4aea-a39f-063119652f9c None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:52.132570 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9438c-0995-4359-8e91-12b26894cd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:52.133253 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9438c-0995-4359-8e91-12b26894cd1d None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:52.133538 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9438c-0995-4359-8e91-12b26894cd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:52.133811 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9438c-0995-4359-8e91-12b26894cd1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:52.199440 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:52.202909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d9438c-0995-4359-8e91-12b26894cd1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:52.204377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d9438c-0995-4359-8e91-12b26894cd1d None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:52.212536 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c0cb3-f4f9-4aea-a39f-063119652f9c None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:52.250477 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104e019-ca82-4f51-be16-e3440ad965bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:52.251378 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e019-ca82-4f51-be16-e3440ad965bd None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:52.251838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e019-ca82-4f51-be16-e3440ad965bd None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:52.252272 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e019-ca82-4f51-be16-e3440ad965bd None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:52.253632 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104e019-ca82-4f51-be16-e3440ad965bd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:52.254334 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104e019-ca82-4f51-be16-e3440ad965bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:52.257888 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104e019-ca82-4f51-be16-e3440ad965bd None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:52.263340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 385/761] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 23 22:27:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:52.264486 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104e019-ca82-4f51-be16-e3440ad965bd None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:52.291723 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:52.312666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 377/762] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:51 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:27:52.313867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9438c-0995-4359-8e91-12b26894cd1d None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:52.378503 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104e019-ca82-4f51-be16-e3440ad965bd None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:52.438246 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:52.442916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9438c-0995-4359-8e91-12b26894cd1d None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:52.450909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c44bc-8eae-4533-86b4-1cdc99fdaef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:52.453360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c44bc-8eae-4533-86b4-1cdc99fdaef9 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:52.454098 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c44bc-8eae-4533-86b4-1cdc99fdaef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:52.455235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c44bc-8eae-4533-86b4-1cdc99fdaef9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:52.466539 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c44bc-8eae-4533-86b4-1cdc99fdaef9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:52.469057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c44bc-8eae-4533-86b4-1cdc99fdaef9 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:52.483761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349fed7-b458-4fa2-8aec-69ebea994ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:52.484426 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349fed7-b458-4fa2-8aec-69ebea994ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:52.484750 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349fed7-b458-4fa2-8aec-69ebea994ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:52.485024 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349fed7-b458-4fa2-8aec-69ebea994ef7 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:52.485735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2349fed7-b458-4fa2-8aec-69ebea994ef7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:52.486041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2349fed7-b458-4fa2-8aec-69ebea994ef7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:52.487401 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2349fed7-b458-4fa2-8aec-69ebea994ef7 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:52.492115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-1d4c44bc-8eae-4533-86b4-1cdc99fdaef9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:52.492115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:52.492115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:52.492115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:52.492115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:52.492115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:52.492115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:52.492115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:52.492115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:52.492115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:52.492115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:52.492115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:52.492115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:52.492115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:52.492115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:52.493329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:52.493329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:52.493329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:52.493329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:52.493329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:52.493329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:52.493329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:52.493329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:52.493329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:52.493329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:52.493329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:52.493329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:52.493329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:52.493329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:52.493329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:52.493329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:52.493329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:52.494507 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:52.494507 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:52.494507 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:52.494507 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:52.494507 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:52.494507 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:52.494507 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:52.494507 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:52.494936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 378/763] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 23 22:27:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:52.494936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349fed7-b458-4fa2-8aec-69ebea994ef7 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:52.497306 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 386/764] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:52.498420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c44bc-8eae-4533-86b4-1cdc99fdaef9 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:52.605765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:52.621649 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349fed7-b458-4fa2-8aec-69ebea994ef7 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:52.630145 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c44bc-8eae-4533-86b4-1cdc99fdaef9 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:52.663431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f22895-cd07-486b-9894-dcfdc18224c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:52.664721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f22895-cd07-486b-9894-dcfdc18224c8 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:52.665270 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f22895-cd07-486b-9894-dcfdc18224c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:52.665774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f22895-cd07-486b-9894-dcfdc18224c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:52.667753 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a26428-5100-4106-b3e3-cc9c5205128b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:52.668415 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a26428-5100-4106-b3e3-cc9c5205128b None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:52.668837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a26428-5100-4106-b3e3-cc9c5205128b None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:52.669074 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a26428-5100-4106-b3e3-cc9c5205128b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:52.669869 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a26428-5100-4106-b3e3-cc9c5205128b None admin] RBAC: Adding query filter params (interface=public, service_id=ca2d2e54bbec44fcb52ac853fb752ee8, region_id=RegionOne) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:52.670173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a26428-5100-4106-b3e3-cc9c5205128b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:52.671538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a26428-5100-4106-b3e3-cc9c5205128b None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:52.679391 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f22895-cd07-486b-9894-dcfdc18224c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:52.680226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 387/765] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 23 22:27:52 2022] GET /identity/v3/endpoints?service_id=ca2d2e54bbec44fcb52ac853fb752ee8&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:52.681155 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a26428-5100-4106-b3e3-cc9c5205128b None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:52.681849 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f22895-cd07-486b-9894-dcfdc18224c8 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:52.700883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-51f22895-cd07-486b-9894-dcfdc18224c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:52.700883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:52.700883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:52.700883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:52.700883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:52.700883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:52.700883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:52.700883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:52.700883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:52.700883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:52.700883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:52.700883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:52.700883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:52.700883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:52.700883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:52.701835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:52.701835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:52.701835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:52.701835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:52.701835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:52.701835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:52.701835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:52.701835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:52.701835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:52.701835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:52.701835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:52.701835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:52.701835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:52.701835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:52.701835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:52.701835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:52.701835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:52.702727 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:52.702727 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:52.702727 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:52.702727 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:52.705044 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 379/766] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:52.706164 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f22895-cd07-486b-9894-dcfdc18224c8 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:52.722917 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:52.789394 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a26428-5100-4106-b3e3-cc9c5205128b None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:52.813953 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f22895-cd07-486b-9894-dcfdc18224c8 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:52.826625 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c010289-5db7-48a6-98a1-17982d7e485c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:52.827365 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c010289-5db7-48a6-98a1-17982d7e485c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:52.827681 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c010289-5db7-48a6-98a1-17982d7e485c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:52.827961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c010289-5db7-48a6-98a1-17982d7e485c None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:52.828781 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c010289-5db7-48a6-98a1-17982d7e485c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:52.829094 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c010289-5db7-48a6-98a1-17982d7e485c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:52.830923 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c010289-5db7-48a6-98a1-17982d7e485c None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:52.848907 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 388/767] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:52.850372 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c010289-5db7-48a6-98a1-17982d7e485c None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:52.853604 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72404324-7908-46d9-93f6-c7c52813a081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:52.854586 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72404324-7908-46d9-93f6-c7c52813a081 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:52.855110 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72404324-7908-46d9-93f6-c7c52813a081 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:52.855592 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72404324-7908-46d9-93f6-c7c52813a081 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:52.856537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72404324-7908-46d9-93f6-c7c52813a081 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:52.858194 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72404324-7908-46d9-93f6-c7c52813a081 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:52.859466 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:52.864439 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 380/768] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 23 22:27:52 2022] GET /identity/v3/services => generated 772 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:52.865964 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72404324-7908-46d9-93f6-c7c52813a081 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:52.955181 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:52.977872 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c010289-5db7-48a6-98a1-17982d7e485c None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:52.981106 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72404324-7908-46d9-93f6-c7c52813a081 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:53.018663 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971e4f5-5dcd-4a00-85dd-3a850bdc6fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:53.019664 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971e4f5-5dcd-4a00-85dd-3a850bdc6fae None admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:53.020021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971e4f5-5dcd-4a00-85dd-3a850bdc6fae None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:53.020377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971e4f5-5dcd-4a00-85dd-3a850bdc6fae None admin] PATH_INFO: `/v3/projects/14265887fea245a5a37abbb3a116e352/users/159cc8befc3a4d57bbd313832084db62/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:53.020501 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7bd90-eeeb-4ebf-9503-a79bd935250b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:53.021929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7bd90-eeeb-4ebf-9503-a79bd935250b None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:53.022422 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7bd90-eeeb-4ebf-9503-a79bd935250b None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:53.022873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7bd90-eeeb-4ebf-9503-a79bd935250b None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:53.023364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7971e4f5-5dcd-4a00-85dd-3a850bdc6fae None admin] RBAC: Authorizing `identity:create_grant(project_id=14265887fea245a5a37abbb3a116e352, user_id=159cc8befc3a4d57bbd313832084db62, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:53.024012 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7bd90-eeeb-4ebf-9503-a79bd935250b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:53.024502 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7bd90-eeeb-4ebf-9503-a79bd935250b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:53.025061 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7971e4f5-5dcd-4a00-85dd-3a850bdc6fae None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:53.026276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7bd90-eeeb-4ebf-9503-a79bd935250b None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:53.039887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 389/769] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:53.041867 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15005692-b47b-42d9-95e7-140bc2ce429b None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:53.042882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15005692-b47b-42d9-95e7-140bc2ce429b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:53.043166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15005692-b47b-42d9-95e7-140bc2ce429b None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:53.044878 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 390/770] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:53.045877 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15005692-b47b-42d9-95e7-140bc2ce429b None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:53.076157 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 381/771] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:27:52 2022] PUT /identity/v3/projects/14265887fea245a5a37abbb3a116e352/users/159cc8befc3a4d57bbd313832084db62/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:27:53.077499 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a531f4-5f78-4b30-a17a-9fcad13d2925 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:53.077804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a531f4-5f78-4b30-a17a-9fcad13d2925 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:53.078137 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a531f4-5f78-4b30-a17a-9fcad13d2925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:53.103518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-f7a531f4-5f78-4b30-a17a-9fcad13d2925 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:53.105752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-f7a531f4-5f78-4b30-a17a-9fcad13d2925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:53.135052 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:53.177106 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15005692-b47b-42d9-95e7-140bc2ce429b None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:53.213500 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 382/772] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:27:53.228943 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12782b1-f268-4473-8b94-379831ae9b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:53.229772 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12782b1-f268-4473-8b94-379831ae9b3d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:53.230056 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12782b1-f268-4473-8b94-379831ae9b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:53.230331 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12782b1-f268-4473-8b94-379831ae9b3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:53.239352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a531f4-5f78-4b30-a17a-9fcad13d2925 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:53.246478 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12782b1-f268-4473-8b94-379831ae9b3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:53.247154 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12782b1-f268-4473-8b94-379831ae9b3d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:53.263882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-a12782b1-f268-4473-8b94-379831ae9b3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:53.263882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:53.263882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:53.263882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:53.263882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:53.263882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:53.263882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:53.263882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:53.263882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:53.263882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:53.263882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:53.263882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:53.263882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:53.263882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:53.263882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:53.265483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:53.265483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:53.265483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:53.265483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:53.265483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:53.265483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:53.265483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:53.265483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:53.265483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:53.265483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:53.265483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:53.265483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:53.265483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:53.265483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:53.265483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:53.265483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:53.265483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:53.266385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:53.266385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:53.266385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:53.266385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:53.266385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:53.266385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:53.266385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:53.266385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:53.266385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 391/773] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:53.271033 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12782b1-f268-4473-8b94-379831ae9b3d None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:53.336531 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:53.372498 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a531f4-5f78-4b30-a17a-9fcad13d2925 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:53.409549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407e58e-2790-4e35-90aa-81c95b6f57ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:53.410286 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407e58e-2790-4e35-90aa-81c95b6f57ef None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:53.410573 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407e58e-2790-4e35-90aa-81c95b6f57ef None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:53.410851 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407e58e-2790-4e35-90aa-81c95b6f57ef None admin] PATH_INFO: `/v3/services/network` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:53.411511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407e58e-2790-4e35-90aa-81c95b6f57ef None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:53.413475 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407e58e-2790-4e35-90aa-81c95b6f57ef None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:53.425729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-5407e58e-2790-4e35-90aa-81c95b6f57ef None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 22:27:53.425729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:53.425729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:27:53.425729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:27:53.425729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:27:53.425729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:27:53.425729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:27:53.425729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:27:53.425729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 22:27:53.425729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:53.425729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:27:53.425729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:53.425729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:53.425729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:53.425729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:53.425729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:53.425729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:53.426735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:53.426735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:53.426735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:53.426735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:53.426735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:53.426735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:53.426735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:27:53.426735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:27:53.426735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:27:53.426735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:27:53.426735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:53.426735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:53.426735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:53.426735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:53.426735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:53.426735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:53.429021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:53.429021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:53.429021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:53.429021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:53.429021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:53.429021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:53.429021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:53.429021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:53.429021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:53.429021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:53.429021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:27:53.429021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:27:53.429021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 22:27:53.429021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:53.429021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 383/774] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:53 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:27:53.434427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407e58e-2790-4e35-90aa-81c95b6f57ef None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:53.442773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12782b1-f268-4473-8b94-379831ae9b3d None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:53.510387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:53.538829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c9944-8bdc-4d69-9183-3306b4e201f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:53.539560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c9944-8bdc-4d69-9183-3306b4e201f2 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:53.539866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c9944-8bdc-4d69-9183-3306b4e201f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:53.540190 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c9944-8bdc-4d69-9183-3306b4e201f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:53.540853 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407e58e-2790-4e35-90aa-81c95b6f57ef None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:53.541609 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c9944-8bdc-4d69-9183-3306b4e201f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:53.541609 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c9944-8bdc-4d69-9183-3306b4e201f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:53.543050 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c9944-8bdc-4d69-9183-3306b4e201f2 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:53.570653 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 392/775] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:53.572957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:53.576218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c9944-8bdc-4d69-9183-3306b4e201f2 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:53.577087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1426d6b-f81f-49b8-b777-49f830d34cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:53.577764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1426d6b-f81f-49b8-b777-49f830d34cfc None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:53.578053 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1426d6b-f81f-49b8-b777-49f830d34cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:53.578333 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1426d6b-f81f-49b8-b777-49f830d34cfc None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:53.579046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1426d6b-f81f-49b8-b777-49f830d34cfc None admin] RBAC: Adding query filter params (name=network) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:53.579356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1426d6b-f81f-49b8-b777-49f830d34cfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:53.582103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1426d6b-f81f-49b8-b777-49f830d34cfc None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:53.585879 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 384/776] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 22:27:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:53.591283 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1426d6b-f81f-49b8-b777-49f830d34cfc None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:53.702234 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1426d6b-f81f-49b8-b777-49f830d34cfc None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:53.738975 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788efa1-d009-4305-9cc4-7c9ebde04360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:53.739652 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788efa1-d009-4305-9cc4-7c9ebde04360 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:53.739941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788efa1-d009-4305-9cc4-7c9ebde04360 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:53.740266 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788efa1-d009-4305-9cc4-7c9ebde04360 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:53.741023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788efa1-d009-4305-9cc4-7c9ebde04360 None admin] RBAC: Adding query filter params (type=network) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:53.741335 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788efa1-d009-4305-9cc4-7c9ebde04360 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:53.742731 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788efa1-d009-4305-9cc4-7c9ebde04360 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:53.748407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 385/777] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 22:27:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:27:53.754138 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788efa1-d009-4305-9cc4-7c9ebde04360 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:53.819010 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:53.830938 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c9944-8bdc-4d69-9183-3306b4e201f2 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:53.859074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788efa1-d009-4305-9cc4-7c9ebde04360 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:53.908558 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806b1ce-9755-4419-9596-35e29a11bc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:53.910559 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806b1ce-9755-4419-9596-35e29a11bc9d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:53.910850 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806b1ce-9755-4419-9596-35e29a11bc9d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:53.911133 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806b1ce-9755-4419-9596-35e29a11bc9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:53.912093 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806b1ce-9755-4419-9596-35e29a11bc9d None admin] RBAC: Adding query filter params (role.id=698ff821321a43d494fe8940cdd08564, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=0a7c204a298c42f0bbc32fa9a6ce1f38) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:53.913027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806b1ce-9755-4419-9596-35e29a11bc9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:53.914489 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806b1ce-9755-4419-9596-35e29a11bc9d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:53.917976 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067a490-0fbb-4cdf-9a70-8ad9d5c893b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:53.918651 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067a490-0fbb-4cdf-9a70-8ad9d5c893b8 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:53.918934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067a490-0fbb-4cdf-9a70-8ad9d5c893b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:53.919213 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067a490-0fbb-4cdf-9a70-8ad9d5c893b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:53.920016 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067a490-0fbb-4cdf-9a70-8ad9d5c893b8 None admin] RBAC: Adding query filter params (interface=public, service_id=6bb4e122d3a747c7aebfe145608b2a97, region_id=RegionOne) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:53.920332 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067a490-0fbb-4cdf-9a70-8ad9d5c893b8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:53.921723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067a490-0fbb-4cdf-9a70-8ad9d5c893b8 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:53.931969 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 393/778] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:27:53 2022] GET /identity/v3/role_assignments?user.id=0a7c204a298c42f0bbc32fa9a6ce1f38&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=698ff821321a43d494fe8940cdd08564 => generated 259 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:53.933920 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 386/779] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 23 22:27:53 2022] GET /identity/v3/endpoints?service_id=6bb4e122d3a747c7aebfe145608b2a97&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:53.939526 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806b1ce-9755-4419-9596-35e29a11bc9d None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:54.006550 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:54.082013 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806b1ce-9755-4419-9596-35e29a11bc9d None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:54.127312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda88027-237e-4d27-8631-ffc9f56103db None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:54.127603 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda88027-237e-4d27-8631-ffc9f56103db None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:54.127891 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda88027-237e-4d27-8631-ffc9f56103db None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:54.129874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 387/780] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:54.135429 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97db3fb-67eb-4516-b20b-12814e28400d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:54.136018 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a9146-9b84-4b86-8778-a92762f3808c None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:54.136142 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97db3fb-67eb-4516-b20b-12814e28400d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:54.136313 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a9146-9b84-4b86-8778-a92762f3808c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:54.136406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97db3fb-67eb-4516-b20b-12814e28400d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:54.136606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a9146-9b84-4b86-8778-a92762f3808c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:54.136718 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97db3fb-67eb-4516-b20b-12814e28400d None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:54.137379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97db3fb-67eb-4516-b20b-12814e28400d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:54.138753 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97db3fb-67eb-4516-b20b-12814e28400d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:54.145129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 394/781] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 23 22:27:53 2022] GET /identity/v3/services => generated 772 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:54.166520 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-783a9146-9b84-4b86-8778-a92762f3808c None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:54.168062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-783a9146-9b84-4b86-8778-a92762f3808c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:54.295191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 388/782] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:27:54.334441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97db3fb-67eb-4516-b20b-12814e28400d None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:54.448097 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97db3fb-67eb-4516-b20b-12814e28400d None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:54.463008 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:54.486929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df916483-1e52-4a9a-887b-c61ac9c9e28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:54.487757 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df916483-1e52-4a9a-887b-c61ac9c9e28c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:54.488045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df916483-1e52-4a9a-887b-c61ac9c9e28c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:54.488324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df916483-1e52-4a9a-887b-c61ac9c9e28c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:54.500768 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df916483-1e52-4a9a-887b-c61ac9c9e28c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:54.508854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-df916483-1e52-4a9a-887b-c61ac9c9e28c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:27:54.508854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:54.508854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:54.508854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:54.508854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:54.508854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:54.508854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:54.508854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:54.508854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:54.508854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:54.508854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:54.508854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:54.508854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:54.508854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:54.508854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:54.508854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:54.509790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:54.509790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:54.509790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:54.509790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:54.509790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:54.509790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:54.509790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:54.509790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:54.509790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:54.509790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:54.509790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:54.509790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:54.509790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:54.509790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:54.509790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:54.509790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:54.510630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:54.510630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:54.510630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:54.510630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:54.510630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:54.510630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:54.510630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:54.510630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:54.510630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:54.510630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:54.510630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:27:54.510630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:54.510630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 395/783] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:27:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:54.516525 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a9146-9b84-4b86-8778-a92762f3808c None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:54.591376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:54.644503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a9146-9b84-4b86-8778-a92762f3808c None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:54.681197 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be540b-18c4-494b-9aa8-f10dbbfbb707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:54.681868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be540b-18c4-494b-9aa8-f10dbbfbb707 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:54.682156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be540b-18c4-494b-9aa8-f10dbbfbb707 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:54.682440 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be540b-18c4-494b-9aa8-f10dbbfbb707 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:54.683186 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be540b-18c4-494b-9aa8-f10dbbfbb707 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:54.683497 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be540b-18c4-494b-9aa8-f10dbbfbb707 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:54.684893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be540b-18c4-494b-9aa8-f10dbbfbb707 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:54.693396 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 389/784] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:54.701649 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df916483-1e52-4a9a-887b-c61ac9c9e28c None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:54.816002 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df916483-1e52-4a9a-887b-c61ac9c9e28c None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:54.852147 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a9655-10ed-42a6-b1e0-a9dc5890edd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:54.852888 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a9655-10ed-42a6-b1e0-a9dc5890edd1 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:54.853178 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a9655-10ed-42a6-b1e0-a9dc5890edd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:54.853453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a9655-10ed-42a6-b1e0-a9dc5890edd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:54.863868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7a9655-10ed-42a6-b1e0-a9dc5890edd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:54.865535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7a9655-10ed-42a6-b1e0-a9dc5890edd1 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:54.887820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-7b7a9655-10ed-42a6-b1e0-a9dc5890edd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:54.887820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:54.887820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:54.887820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:54.887820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:54.887820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:54.887820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:54.887820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:54.887820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:54.887820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:54.887820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:54.887820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:54.887820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:54.887820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:54.887820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:54.888792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:54.888792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:54.888792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:54.888792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:54.888792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:54.888792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:54.888792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:54.888792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:54.888792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:54.888792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:54.888792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:54.888792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:54.888792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:54.888792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:54.888792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:54.888792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:54.888792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:54.889733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:54.889733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:54.889733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:54.889733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:54.889733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 396/785] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:54.894639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be540b-18c4-494b-9aa8-f10dbbfbb707 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:55.031557 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be540b-18c4-494b-9aa8-f10dbbfbb707 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:55.052013 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:55.077566 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e588a9-9c04-4225-8a8a-b54c3afe02f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:55.078594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e588a9-9c04-4225-8a8a-b54c3afe02f2 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:55.079079 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e588a9-9c04-4225-8a8a-b54c3afe02f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:55.079606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e588a9-9c04-4225-8a8a-b54c3afe02f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:55.080537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e588a9-9c04-4225-8a8a-b54c3afe02f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:55.081084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e588a9-9c04-4225-8a8a-b54c3afe02f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:55.082637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e588a9-9c04-4225-8a8a-b54c3afe02f2 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:55.101496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 390/786] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:55.107190 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a9655-10ed-42a6-b1e0-a9dc5890edd1 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:55.158417 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:55.256773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a9655-10ed-42a6-b1e0-a9dc5890edd1 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:55.296562 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8171fb-3f59-4f0e-bfcf-8c966cc04c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:55.297461 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8171fb-3f59-4f0e-bfcf-8c966cc04c39 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:55.297748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8171fb-3f59-4f0e-bfcf-8c966cc04c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:55.298028 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8171fb-3f59-4f0e-bfcf-8c966cc04c39 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:55.307790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8171fb-3f59-4f0e-bfcf-8c966cc04c39 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:55.309283 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8171fb-3f59-4f0e-bfcf-8c966cc04c39 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:55.324447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-ae8171fb-3f59-4f0e-bfcf-8c966cc04c39 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:27:55.324447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:55.324447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:55.324447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:55.324447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:55.324447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:55.324447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:55.324447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:55.324447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:55.324447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:55.324447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:55.324447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:55.324447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:55.324447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:55.324447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:55.324447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:55.325463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:55.325463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:55.325463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:55.325463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:55.325463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:55.325463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:55.325463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:55.325463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:55.325463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:55.325463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:55.325463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:55.325463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:55.325463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:55.325463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:55.325463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:55.325463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:55.325463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:55.325463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:55.326434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:55.326434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:55.326434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:55.326434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:55.326434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:55.326434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:55.326434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:55.326434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:55.326434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:27:55.326434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:55.326434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 397/787] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:27:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:55.331380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e588a9-9c04-4225-8a8a-b54c3afe02f2 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:55.482888 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e588a9-9c04-4225-8a8a-b54c3afe02f2 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:55.492842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0e3a2-a427-499a-ad58-89ad2c2a0588 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:55.493331 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0e3a2-a427-499a-ad58-89ad2c2a0588 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:55.493407 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0e3a2-a427-499a-ad58-89ad2c2a0588 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:55.496171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 398/788] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:55.500346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f00ce7-eea3-42cf-9eeb-8db0f62d58a7 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:55.500654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f00ce7-eea3-42cf-9eeb-8db0f62d58a7 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:55.500940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f00ce7-eea3-42cf-9eeb-8db0f62d58a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:55.529999 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d412ec-0772-4870-8820-469fdc96a1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:55.530984 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d412ec-0772-4870-8820-469fdc96a1ab None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:55.531445 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d412ec-0772-4870-8820-469fdc96a1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:55.531930 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d412ec-0772-4870-8820-469fdc96a1ab None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:55.532972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d412ec-0772-4870-8820-469fdc96a1ab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:55.533459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d412ec-0772-4870-8820-469fdc96a1ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:55.535379 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d412ec-0772-4870-8820-469fdc96a1ab None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:55.541939 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-a8f00ce7-eea3-42cf-9eeb-8db0f62d58a7 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:55.543095 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a8f00ce7-eea3-42cf-9eeb-8db0f62d58a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:55.556471 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 391/789] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 23 22:27:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:55.561468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d412ec-0772-4870-8820-469fdc96a1ab None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:55.604966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:55.672674 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 399/790] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:27:55.674339 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2773dc-6a91-4357-903f-8bec7c77b1e3 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:55.674728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2773dc-6a91-4357-903f-8bec7c77b1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:55.675127 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2773dc-6a91-4357-903f-8bec7c77b1e3 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:55.676669 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d412ec-0772-4870-8820-469fdc96a1ab None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:55.681209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 400/791] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:55.682494 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6fec4-06ed-410e-b62b-2346ce482b8d None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:55.682786 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6fec4-06ed-410e-b62b-2346ce482b8d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:55.683069 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6fec4-06ed-410e-b62b-2346ce482b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:55.700327 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-0ec6fec4-06ed-410e-b62b-2346ce482b8d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:55.701678 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-0ec6fec4-06ed-410e-b62b-2346ce482b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:55.713356 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:55.714151 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6589dc-061f-42a3-8643-05f99fb36ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:55.714994 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6589dc-061f-42a3-8643-05f99fb36ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:55.715282 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6589dc-061f-42a3-8643-05f99fb36ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:55.715555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6589dc-061f-42a3-8643-05f99fb36ba6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:55.727598 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6589dc-061f-42a3-8643-05f99fb36ba6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:55.729153 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6589dc-061f-42a3-8643-05f99fb36ba6 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:55.747786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-2d6589dc-061f-42a3-8643-05f99fb36ba6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:55.747786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:55.747786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:55.747786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:55.747786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:55.747786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:55.747786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:55.747786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:55.747786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:55.747786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:55.747786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:55.747786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:55.747786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:55.747786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:55.747786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:55.748957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:55.748957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:55.748957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:55.748957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:55.748957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:55.748957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:55.748957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:55.748957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:55.748957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:55.748957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:55.748957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:55.748957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:55.748957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:55.748957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:55.748957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:55.748957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:55.748957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:55.749853 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:55.749853 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:55.749853 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:55.749853 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:55.749853 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 392/792] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:55.749853 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6589dc-061f-42a3-8643-05f99fb36ba6 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:55.774978 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 401/793] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:27:55.780901 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6fec4-06ed-410e-b62b-2346ce482b8d None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:55.851991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:55.869461 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6589dc-061f-42a3-8643-05f99fb36ba6 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:55.914443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f93bab-f3b0-4999-b52f-4449343de080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:55.915406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f93bab-f3b0-4999-b52f-4449343de080 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:55.915760 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f93bab-f3b0-4999-b52f-4449343de080 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:55.916681 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f93bab-f3b0-4999-b52f-4449343de080 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:55.917826 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f93bab-f3b0-4999-b52f-4449343de080 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:55.919521 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f93bab-f3b0-4999-b52f-4449343de080 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:55.923676 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6fec4-06ed-410e-b62b-2346ce482b8d None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:55.936370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-11f93bab-f3b0-4999-b52f-4449343de080 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 22:27:55.936370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:55.936370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:27:55.936370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:27:55.936370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:27:55.936370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:27:55.936370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:27:55.936370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:27:55.936370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 22:27:55.936370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:55.936370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:27:55.936370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:55.936370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:55.936370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:55.936370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:55.936370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:55.937577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:55.937577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:55.937577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:55.937577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:55.937577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:55.937577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:55.937577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:55.937577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:27:55.937577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:27:55.937577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:27:55.937577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:27:55.937577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:55.937577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:55.937577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:55.937577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:55.937577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:55.938601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:55.938601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:55.938601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:55.938601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:55.938601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:55.938601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:55.938601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:55.938601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:55.938601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:55.938601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:55.938601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:55.938601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:27:55.938601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:27:55.938601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 22:27:55.938601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:55.940654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 393/794] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Jun 23 22:27:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:55.941719 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f93bab-f3b0-4999-b52f-4449343de080 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:55.977637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c52a2-53ad-4ef3-a485-bc7e723bd1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:55.978315 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c52a2-53ad-4ef3-a485-bc7e723bd1ff None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:55.978993 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c52a2-53ad-4ef3-a485-bc7e723bd1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:55.979290 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c52a2-53ad-4ef3-a485-bc7e723bd1ff None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:55.980294 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c52a2-53ad-4ef3-a485-bc7e723bd1ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:55.980632 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c52a2-53ad-4ef3-a485-bc7e723bd1ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:55.982023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c52a2-53ad-4ef3-a485-bc7e723bd1ff None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:56.000723 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 402/795] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:56.001865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c52a2-53ad-4ef3-a485-bc7e723bd1ff None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:56.014464 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:56.062211 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f93bab-f3b0-4999-b52f-4449343de080 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:56.100449 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01da64-9d49-4f3a-88b3-0dd23c7df507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:56.101726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01da64-9d49-4f3a-88b3-0dd23c7df507 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:56.102223 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01da64-9d49-4f3a-88b3-0dd23c7df507 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:56.102689 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01da64-9d49-4f3a-88b3-0dd23c7df507 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:56.113934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01da64-9d49-4f3a-88b3-0dd23c7df507 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:56.123152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-2c01da64-9d49-4f3a-88b3-0dd23c7df507 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:27:56.123152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:56.123152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:56.123152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:56.123152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:56.123152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:56.123152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:56.123152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:56.123152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:56.123152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:56.123152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:56.123152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:56.123152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:56.123152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:56.123152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:56.123152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:56.124093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:56.124093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:56.124093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:56.124093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:56.124093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:56.124093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:56.124093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:56.124093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:56.124093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:56.124093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:56.124093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:56.124093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:56.124093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:56.124093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:56.124093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:56.124093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:56.124954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:56.124954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:56.124954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:56.124954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:56.124954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:56.124954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:56.124954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:56.124954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:56.124954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:56.124954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:56.124954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:27:56.124954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:56.127689 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 394/796] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:27:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:56.129069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01da64-9d49-4f3a-88b3-0dd23c7df507 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:56.137283 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c52a2-53ad-4ef3-a485-bc7e723bd1ff None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:56.144263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:56.182405 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936388a2-81fb-4da8-b33e-d4d86d07b87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:56.183082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936388a2-81fb-4da8-b33e-d4d86d07b87c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:56.183732 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936388a2-81fb-4da8-b33e-d4d86d07b87c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:56.184034 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936388a2-81fb-4da8-b33e-d4d86d07b87c None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:56.184797 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936388a2-81fb-4da8-b33e-d4d86d07b87c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:56.185360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936388a2-81fb-4da8-b33e-d4d86d07b87c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:56.186899 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936388a2-81fb-4da8-b33e-d4d86d07b87c None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:56.196646 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 403/797] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 23 22:27:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:56.197872 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936388a2-81fb-4da8-b33e-d4d86d07b87c None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:56.262656 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01da64-9d49-4f3a-88b3-0dd23c7df507 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:56.281876 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:56.306660 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759bf15-4f35-4241-afff-c3707420d5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:56.307736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759bf15-4f35-4241-afff-c3707420d5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:56.308021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759bf15-4f35-4241-afff-c3707420d5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:56.308450 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759bf15-4f35-4241-afff-c3707420d5a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:56.314534 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936388a2-81fb-4da8-b33e-d4d86d07b87c None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:56.317799 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759bf15-4f35-4241-afff-c3707420d5a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:56.319843 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759bf15-4f35-4241-afff-c3707420d5a0 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:56.340085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-2759bf15-4f35-4241-afff-c3707420d5a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:56.340085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:56.340085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:56.340085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:56.340085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:56.340085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:56.340085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:56.340085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:56.340085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:56.340085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:56.340085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:56.340085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:56.340085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:56.340085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:56.340085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:56.341050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:56.341050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:56.341050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:56.341050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:56.341050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:56.341050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:56.341050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:56.341050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:56.341050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:56.341050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:56.341050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:56.341050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:56.341050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:56.341050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:56.341050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:56.341050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:56.341050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:56.342003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:56.342003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:56.342003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:56.342003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:56.342003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:56.342003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:56.342003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:56.342003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:56.342926 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 395/798] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:56.344846 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759bf15-4f35-4241-afff-c3707420d5a0 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:56.361777 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f36eb2-6eb8-4b24-a505-154a884f7922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:56.362717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f36eb2-6eb8-4b24-a505-154a884f7922 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:56.363004 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f36eb2-6eb8-4b24-a505-154a884f7922 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:56.363805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f36eb2-6eb8-4b24-a505-154a884f7922 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:56.364901 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f36eb2-6eb8-4b24-a505-154a884f7922 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:56.365260 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f36eb2-6eb8-4b24-a505-154a884f7922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:56.367291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f36eb2-6eb8-4b24-a505-154a884f7922 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:56.379495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 404/799] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:56.381480 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f36eb2-6eb8-4b24-a505-154a884f7922 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:56.422296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:56.456310 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759bf15-4f35-4241-afff-c3707420d5a0 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:56.501762 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d9f32-6c31-4f5b-9a82-4b2d379a2735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:56.502694 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d9f32-6c31-4f5b-9a82-4b2d379a2735 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:56.502985 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d9f32-6c31-4f5b-9a82-4b2d379a2735 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:56.503685 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d9f32-6c31-4f5b-9a82-4b2d379a2735 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:56.504700 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d9f32-6c31-4f5b-9a82-4b2d379a2735 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:56.505016 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d9f32-6c31-4f5b-9a82-4b2d379a2735 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:56.507823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d9f32-6c31-4f5b-9a82-4b2d379a2735 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:56.512989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f36eb2-6eb8-4b24-a505-154a884f7922 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:56.516666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 396/800] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 23 22:27:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:56.517507 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d9f32-6c31-4f5b-9a82-4b2d379a2735 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:56.554847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:56.560362 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1494f8b9-9a17-414d-b3a2-f5338c68a7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:56.561057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494f8b9-9a17-414d-b3a2-f5338c68a7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:56.561343 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494f8b9-9a17-414d-b3a2-f5338c68a7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:56.561621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494f8b9-9a17-414d-b3a2-f5338c68a7b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:56.562411 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1494f8b9-9a17-414d-b3a2-f5338c68a7b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:56.562716 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1494f8b9-9a17-414d-b3a2-f5338c68a7b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:56.564217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1494f8b9-9a17-414d-b3a2-f5338c68a7b7 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:56.577772 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 405/801] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:56.578390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1494f8b9-9a17-414d-b3a2-f5338c68a7b7 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:56.630861 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d9f32-6c31-4f5b-9a82-4b2d379a2735 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:56.665425 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6724483-2b63-4eee-a2ba-b1561b4ef7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:56.666479 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6724483-2b63-4eee-a2ba-b1561b4ef7b6 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:56.666769 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6724483-2b63-4eee-a2ba-b1561b4ef7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:56.667047 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6724483-2b63-4eee-a2ba-b1561b4ef7b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:56.668996 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:56.678825 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6724483-2b63-4eee-a2ba-b1561b4ef7b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:56.682950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6724483-2b63-4eee-a2ba-b1561b4ef7b6 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:56.685459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1494f8b9-9a17-414d-b3a2-f5338c68a7b7 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:56.697648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-b6724483-2b63-4eee-a2ba-b1561b4ef7b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:56.697648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:56.697648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:56.697648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:56.697648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:56.697648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:56.697648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:56.697648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:56.697648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:56.697648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:56.697648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:56.697648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:56.697648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:56.697648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:56.697648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:56.698723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:56.698723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:56.698723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:56.698723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:56.698723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:56.698723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:56.698723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:56.698723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:56.698723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:56.698723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:56.698723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:56.698723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:56.698723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:56.698723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:56.698723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:56.698723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:56.698723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:56.699647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:56.699647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:56.699647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:56.699647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:56.699647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 397/802] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:56.700481 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76cd84b-d503-45e8-9d8f-ab0abdbd67ec None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:56.700876 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76cd84b-d503-45e8-9d8f-ab0abdbd67ec None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:56.701161 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76cd84b-d503-45e8-9d8f-ab0abdbd67ec None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:56.702538 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 398/803] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:56.703080 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76cd84b-d503-45e8-9d8f-ab0abdbd67ec None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:56.729263 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233f737-a692-4045-9beb-516327628a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:56.729999 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233f737-a692-4045-9beb-516327628a1f None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:56.730299 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233f737-a692-4045-9beb-516327628a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:56.730581 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233f737-a692-4045-9beb-516327628a1f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:56.731448 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b233f737-a692-4045-9beb-516327628a1f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:56.732723 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b233f737-a692-4045-9beb-516327628a1f None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:56.809382 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76cd84b-d503-45e8-9d8f-ab0abdbd67ec None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:56.833680 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 406/804] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 23 22:27:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:27:56.834851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233f737-a692-4045-9beb-516327628a1f None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:56.853954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b83e70-44e7-4e5a-86b5-008aa61cd025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:56.854869 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b83e70-44e7-4e5a-86b5-008aa61cd025 None admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:56.855152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b83e70-44e7-4e5a-86b5-008aa61cd025 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:56.855429 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b83e70-44e7-4e5a-86b5-008aa61cd025 None admin] PATH_INFO: `/v3/projects/14265887fea245a5a37abbb3a116e352/users/b321586cdb3d4d25904472aaa4adba71/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:56.857575 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b83e70-44e7-4e5a-86b5-008aa61cd025 None admin] RBAC: Authorizing `identity:create_grant(project_id=14265887fea245a5a37abbb3a116e352, user_id=b321586cdb3d4d25904472aaa4adba71, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:56.859060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b83e70-44e7-4e5a-86b5-008aa61cd025 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:56.891641 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:56.926530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 399/805] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:27:56 2022] PUT /identity/v3/projects/14265887fea245a5a37abbb3a116e352/users/b321586cdb3d4d25904472aaa4adba71/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:27:56.929105 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856cd1b0-d457-422f-a453-b14123e1bf3d None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:56.932820 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856cd1b0-d457-422f-a453-b14123e1bf3d None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:56.932820 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856cd1b0-d457-422f-a453-b14123e1bf3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:56.959963 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-856cd1b0-d457-422f-a453-b14123e1bf3d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:56.960401 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233f737-a692-4045-9beb-516327628a1f None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:56.961991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-856cd1b0-d457-422f-a453-b14123e1bf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:57.007631 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907b0b5-2ec1-4c06-a881-d98e4b21233b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:57.008438 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907b0b5-2ec1-4c06-a881-d98e4b21233b None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:57.008857 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907b0b5-2ec1-4c06-a881-d98e4b21233b None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:57.009316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907b0b5-2ec1-4c06-a881-d98e4b21233b None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:57.010498 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907b0b5-2ec1-4c06-a881-d98e4b21233b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:57.010945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907b0b5-2ec1-4c06-a881-d98e4b21233b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:57.012249 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907b0b5-2ec1-4c06-a881-d98e4b21233b None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:57.020505 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:57.026262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 407/806] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:57.028050 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cde1a-ad4b-4565-b0f6-8d621ec91587 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:57.028050 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cde1a-ad4b-4565-b0f6-8d621ec91587 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:57.028050 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cde1a-ad4b-4565-b0f6-8d621ec91587 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:57.030694 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 408/807] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:57.033450 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6cde1a-ad4b-4565-b0f6-8d621ec91587 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:57.074152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 400/808] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:27:57.075603 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266d7d4-f0e3-49d8-be88-682debdb8775 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:57.075897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266d7d4-f0e3-49d8-be88-682debdb8775 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:57.076182 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266d7d4-f0e3-49d8-be88-682debdb8775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:57.094663 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-f266d7d4-f0e3-49d8-be88-682debdb8775 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:57.096003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-f266d7d4-f0e3-49d8-be88-682debdb8775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:57.168680 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6cde1a-ad4b-4565-b0f6-8d621ec91587 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:57.179218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:57.193560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 401/809] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:27:57.194100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266d7d4-f0e3-49d8-be88-682debdb8775 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:57.226227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780025a-9a0b-40a9-a33f-56f93f568f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:57.227035 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780025a-9a0b-40a9-a33f-56f93f568f98 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:57.227323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780025a-9a0b-40a9-a33f-56f93f568f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:57.227612 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780025a-9a0b-40a9-a33f-56f93f568f98 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:57.238330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2780025a-9a0b-40a9-a33f-56f93f568f98 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:57.239793 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2780025a-9a0b-40a9-a33f-56f93f568f98 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:57.258577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-2780025a-9a0b-40a9-a33f-56f93f568f98 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:27:57.258577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:57.258577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:57.258577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:57.258577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:57.258577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:57.258577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:57.258577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:57.258577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:57.258577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:57.258577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:57.258577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:57.258577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:57.258577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:57.258577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:57.258577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:57.259798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:57.259798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:57.259798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:57.259798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:57.259798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:57.259798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:57.259798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:57.259798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:57.259798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:57.259798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:57.259798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:57.259798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:57.259798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:57.259798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:57.259798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:57.259798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:57.259798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:57.259798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:57.261109 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:57.261109 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:57.261109 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:57.261109 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:57.261109 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:57.261109 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:57.261109 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:57.261109 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:57.261109 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:27:57.261109 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:57.261109 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 409/810] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 23 22:27:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:27:57.265011 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780025a-9a0b-40a9-a33f-56f93f568f98 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:57.344113 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266d7d4-f0e3-49d8-be88-682debdb8775 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:57.345923 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:57.381291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780025a-9a0b-40a9-a33f-56f93f568f98 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:57.384636 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f2a5e-9d30-4fc7-9910-6818d8b91b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:57.385454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f2a5e-9d30-4fc7-9910-6818d8b91b6a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:57.385744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f2a5e-9d30-4fc7-9910-6818d8b91b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:57.386026 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f2a5e-9d30-4fc7-9910-6818d8b91b6a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:57.399500 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f2a5e-9d30-4fc7-9910-6818d8b91b6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:57.407526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-067f2a5e-9d30-4fc7-9910-6818d8b91b6a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:57.407526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:57.407526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:57.407526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:57.407526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:57.407526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:57.407526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:57.407526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:57.407526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:57.407526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:57.407526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:57.407526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:57.407526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:57.407526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:57.407526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:57.407526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:57.408491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:57.408491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:57.408491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:57.408491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:57.408491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:57.408491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:57.408491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:57.408491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:57.408491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:57.408491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:57.408491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:57.408491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:57.408491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:57.408491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:57.408491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:57.408491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:57.409409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:57.409409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:57.409409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:57.409409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:57.409409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:57.409409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:57.409409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:57.409409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:57.409409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:57.409409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:57.409409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:27:57.409409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:57.409409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 402/811] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:27:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:57.410119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f2a5e-9d30-4fc7-9910-6818d8b91b6a None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:57.419934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309e2de-f224-431f-bd5d-e51aad5a1d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:57.420787 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309e2de-f224-431f-bd5d-e51aad5a1d68 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:57.421080 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309e2de-f224-431f-bd5d-e51aad5a1d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:57.421356 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309e2de-f224-431f-bd5d-e51aad5a1d68 None admin] PATH_INFO: `/v3/services/network` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:57.422019 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309e2de-f224-431f-bd5d-e51aad5a1d68 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:57.423407 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309e2de-f224-431f-bd5d-e51aad5a1d68 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:57.435879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-0309e2de-f224-431f-bd5d-e51aad5a1d68 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 22:27:57.435879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:57.435879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:27:57.435879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:27:57.435879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:27:57.435879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:27:57.435879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:27:57.435879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:27:57.435879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 22:27:57.435879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:57.435879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:27:57.435879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:57.435879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:57.435879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:57.435879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:57.435879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:57.435879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:57.437203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:57.437203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:57.437203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:57.437203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:57.437203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:57.437203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:57.437203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:27:57.437203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:27:57.437203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:27:57.437203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:27:57.437203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:57.437203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:57.437203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:57.437203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:57.437203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:57.437203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:57.438038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:57.438038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:57.438038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:57.438038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:57.438038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:57.438038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:57.438038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:57.438038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:57.438038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:57.438038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:57.438038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:27:57.438038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:27:57.438038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 22:27:57.438038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:57.438038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 410/812] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:57 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:57.439320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309e2de-f224-431f-bd5d-e51aad5a1d68 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:57.471614 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:57.535122 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f2a5e-9d30-4fc7-9910-6818d8b91b6a None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:57.545899 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309e2de-f224-431f-bd5d-e51aad5a1d68 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:57.577313 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54546f87-1011-4d36-b7b6-31affe5c2d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:57.577998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54546f87-1011-4d36-b7b6-31affe5c2d92 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:57.578282 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54546f87-1011-4d36-b7b6-31affe5c2d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:57.578561 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54546f87-1011-4d36-b7b6-31affe5c2d92 None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:57.579410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54546f87-1011-4d36-b7b6-31affe5c2d92 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:57.579725 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54546f87-1011-4d36-b7b6-31affe5c2d92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:57.581206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54546f87-1011-4d36-b7b6-31affe5c2d92 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:57.598371 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2919fd-98c5-4775-8b0f-35018eba22f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:57.598974 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2919fd-98c5-4775-8b0f-35018eba22f6 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:57.599290 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2919fd-98c5-4775-8b0f-35018eba22f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:57.599648 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2919fd-98c5-4775-8b0f-35018eba22f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:57.600405 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2919fd-98c5-4775-8b0f-35018eba22f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:57.600774 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2919fd-98c5-4775-8b0f-35018eba22f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:57.601368 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 403/813] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 23 22:27:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:57.602544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54546f87-1011-4d36-b7b6-31affe5c2d92 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:57.607724 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2919fd-98c5-4775-8b0f-35018eba22f6 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:57.619113 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 411/814] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:27:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:57.619956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2919fd-98c5-4775-8b0f-35018eba22f6 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:57.630022 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:57.726299 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54546f87-1011-4d36-b7b6-31affe5c2d92 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:57.733700 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:57.742558 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2919fd-98c5-4775-8b0f-35018eba22f6 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:57.763926 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311affa3-964e-44ca-a748-fdc3ab60a0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:57.764621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311affa3-964e-44ca-a748-fdc3ab60a0df None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:57.764927 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311affa3-964e-44ca-a748-fdc3ab60a0df None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:57.765210 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311affa3-964e-44ca-a748-fdc3ab60a0df None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:57.766223 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311affa3-964e-44ca-a748-fdc3ab60a0df None admin] RBAC: Adding query filter params (name=network) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:57.766223 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311affa3-964e-44ca-a748-fdc3ab60a0df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:57.767602 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311affa3-964e-44ca-a748-fdc3ab60a0df None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:57.772768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 404/815] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 22:27:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:57.773942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311affa3-964e-44ca-a748-fdc3ab60a0df None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:57.787279 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d3e7d-a383-4157-a750-c0a8352e30d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:57.788763 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d3e7d-a383-4157-a750-c0a8352e30d4 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:57.789279 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d3e7d-a383-4157-a750-c0a8352e30d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:57.789894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d3e7d-a383-4157-a750-c0a8352e30d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:57.861885 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d3e7d-a383-4157-a750-c0a8352e30d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:57.863678 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d3e7d-a383-4157-a750-c0a8352e30d4 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:57.867836 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:57.884994 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311affa3-964e-44ca-a748-fdc3ab60a0df None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:57.923106 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4678c8-e0fe-44c4-8587-531fb2577659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:57.924158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4678c8-e0fe-44c4-8587-531fb2577659 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:57.924736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4678c8-e0fe-44c4-8587-531fb2577659 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:57.925175 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4678c8-e0fe-44c4-8587-531fb2577659 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:57.937560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4678c8-e0fe-44c4-8587-531fb2577659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:57.939225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4678c8-e0fe-44c4-8587-531fb2577659 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:57.959595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-be4678c8-e0fe-44c4-8587-531fb2577659 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:57.959595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:57.959595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:57.959595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:57.959595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:57.959595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:57.959595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:57.959595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:57.959595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:57.959595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:57.959595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:57.959595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:57.959595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:57.959595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:57.959595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:57.960752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:57.960752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:57.960752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:57.960752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:57.960752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:57.960752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:57.960752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:57.960752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:57.960752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:57.960752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:57.960752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:57.960752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:57.960752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:57.960752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:57.960752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:57.960752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:57.960752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:57.961661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:57.961661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:57.961661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:57.961661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:57.963504 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:57.964780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 405/816] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:57.965716 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4678c8-e0fe-44c4-8587-531fb2577659 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:57.976175 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 412/817] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:27:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:27:57.976175 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d3e7d-a383-4157-a750-c0a8352e30d4 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:58.081459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d3e7d-a383-4157-a750-c0a8352e30d4 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:58.084569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4678c8-e0fe-44c4-8587-531fb2577659 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:58.119681 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a9d94-c20d-483d-b940-106c8bedf97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:58.120276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a9d94-c20d-483d-b940-106c8bedf97d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:58.120646 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a9d94-c20d-483d-b940-106c8bedf97d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:58.120876 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a9d94-c20d-483d-b940-106c8bedf97d None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:58.121721 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a9d94-c20d-483d-b940-106c8bedf97d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:58.122038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a9d94-c20d-483d-b940-106c8bedf97d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:58.123412 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a9d94-c20d-483d-b940-106c8bedf97d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:58.125732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f490cc9-70af-45b2-bacf-d692662f6e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:58.126705 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f490cc9-70af-45b2-bacf-d692662f6e89 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:58.127192 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f490cc9-70af-45b2-bacf-d692662f6e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:58.127668 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f490cc9-70af-45b2-bacf-d692662f6e89 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:58.128591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f490cc9-70af-45b2-bacf-d692662f6e89 None admin] RBAC: Adding query filter params (type=network) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:58.129122 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f490cc9-70af-45b2-bacf-d692662f6e89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:58.130887 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f490cc9-70af-45b2-bacf-d692662f6e89 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:58.133544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:58.136793 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 413/818] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:58.137774 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a9d94-c20d-483d-b940-106c8bedf97d None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:58.143244 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 406/819] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 22:27:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:58.144683 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f490cc9-70af-45b2-bacf-d692662f6e89 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:58.199663 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:58.248179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a9d94-c20d-483d-b940-106c8bedf97d None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:58.260690 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f490cc9-70af-45b2-bacf-d692662f6e89 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:58.295650 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415cbab-7e25-4bd3-b85e-9ebb5e339ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:58.296463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415cbab-7e25-4bd3-b85e-9ebb5e339ffc None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:58.296794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415cbab-7e25-4bd3-b85e-9ebb5e339ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:58.297080 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415cbab-7e25-4bd3-b85e-9ebb5e339ffc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:58.299765 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b1017-f060-45bc-8ef0-13baf76fd7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:58.300938 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b1017-f060-45bc-8ef0-13baf76fd7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:58.301443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b1017-f060-45bc-8ef0-13baf76fd7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:58.301919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b1017-f060-45bc-8ef0-13baf76fd7a5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:58.309392 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415cbab-7e25-4bd3-b85e-9ebb5e339ffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:58.310864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415cbab-7e25-4bd3-b85e-9ebb5e339ffc None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:58.311442 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b1017-f060-45bc-8ef0-13baf76fd7a5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:58.313214 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b1017-f060-45bc-8ef0-13baf76fd7a5 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:58.329184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-8415cbab-7e25-4bd3-b85e-9ebb5e339ffc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:58.329184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:58.329184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:58.329184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:58.329184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:58.329184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:58.329184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:58.329184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:58.329184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:58.329184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:58.329184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:58.329184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:58.329184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:58.329184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:58.329184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:58.338323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:58.338323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:58.338323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:58.338323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:58.338323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:58.338323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:58.338323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:58.338323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:58.338323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:58.338323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:58.338323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:58.338323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:58.338323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:58.338323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:58.338323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:58.338323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:58.338323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:58.339223 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:58.339223 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:58.339223 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:58.339223 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:58.339454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-496b1017-f060-45bc-8ef0-13baf76fd7a5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:27:58.339454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:58.339454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:58.339454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:58.339454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:58.339454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:58.339454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:58.339454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:58.339454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:58.339454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:58.339454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:58.339454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:58.339454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:27:58.339454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:27:58.339454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:27:58.339454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:27:58.340364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:58.340364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:58.340364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:27:58.340364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:58.340364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:27:58.340364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:27:58.340364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:58.340364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:58.340364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:58.340364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:58.340364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:58.340364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:58.340364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:58.340364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:58.340364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:58.340364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:58.340364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:58.340364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:58.341326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:58.341326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:58.341326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:27:58.341326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:27:58.341326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:27:58.341326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:27:58.341326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:27:58.341326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:27:58.341326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:27:58.341326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:58.343144 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 414/820] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:58.343144 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415cbab-7e25-4bd3-b85e-9ebb5e339ffc None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:58.343388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 407/821] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 23 22:27:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:58.343388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b1017-f060-45bc-8ef0-13baf76fd7a5 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:58.407492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:58.448505 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415cbab-7e25-4bd3-b85e-9ebb5e339ffc None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:58.454067 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b1017-f060-45bc-8ef0-13baf76fd7a5 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:58.455798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:58.485233 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39eb03-76c5-4ad1-a82d-02cdf97bc027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:58.485900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39eb03-76c5-4ad1-a82d-02cdf97bc027 None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:58.486192 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39eb03-76c5-4ad1-a82d-02cdf97bc027 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:58.486484 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39eb03-76c5-4ad1-a82d-02cdf97bc027 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:58.487257 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc39eb03-76c5-4ad1-a82d-02cdf97bc027 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:58.488699 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc39eb03-76c5-4ad1-a82d-02cdf97bc027 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:58.493525 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab985e5-174b-409b-ab6d-e1bcd0adb245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:58.494389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab985e5-174b-409b-ab6d-e1bcd0adb245 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:58.495595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab985e5-174b-409b-ab6d-e1bcd0adb245 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:58.496326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab985e5-174b-409b-ab6d-e1bcd0adb245 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:58.497325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab985e5-174b-409b-ab6d-e1bcd0adb245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:58.497801 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab985e5-174b-409b-ab6d-e1bcd0adb245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:58.499448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab985e5-174b-409b-ab6d-e1bcd0adb245 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:58.512612 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 408/822] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:58.516629 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab985e5-174b-409b-ab6d-e1bcd0adb245 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:58.568603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 415/823] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 23 22:27:58 2022] POST /identity/v3/endpoints => generated 327 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:27:58.568603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39eb03-76c5-4ad1-a82d-02cdf97bc027 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:58.743120 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab985e5-174b-409b-ab6d-e1bcd0adb245 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:58.772789 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:58.803133 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46e8f7-0d48-48f0-aea4-2901d3f1d2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:58.804215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46e8f7-0d48-48f0-aea4-2901d3f1d2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:58.804773 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46e8f7-0d48-48f0-aea4-2901d3f1d2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:58.805274 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46e8f7-0d48-48f0-aea4-2901d3f1d2a1 None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:58.806934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46e8f7-0d48-48f0-aea4-2901d3f1d2a1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:58.807446 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46e8f7-0d48-48f0-aea4-2901d3f1d2a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:58.810284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46e8f7-0d48-48f0-aea4-2901d3f1d2a1 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:58.835266 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 409/824] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 23 22:27:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:58.844029 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46e8f7-0d48-48f0-aea4-2901d3f1d2a1 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:58.890103 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39eb03-76c5-4ad1-a82d-02cdf97bc027 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:58.937717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dc679-029c-44f4-b0a3-6958eb395c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:58.938737 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dc679-029c-44f4-b0a3-6958eb395c3f None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:58.939190 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dc679-029c-44f4-b0a3-6958eb395c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:58.939571 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dc679-029c-44f4-b0a3-6958eb395c3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:58.948367 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dc679-029c-44f4-b0a3-6958eb395c3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:58.950355 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dc679-029c-44f4-b0a3-6958eb395c3f None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:58.968771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-f27dc679-029c-44f4-b0a3-6958eb395c3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:58.968771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:58.968771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:58.968771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:58.968771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:58.968771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:58.968771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:58.968771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:58.968771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:58.968771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:58.968771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:58.968771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:58.968771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:58.968771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:58.968771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:58.969909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:58.969909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:58.969909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:58.969909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:58.969909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:58.969909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:58.969909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:58.969909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:58.969909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:58.969909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:58.969909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:58.969909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:58.969909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:58.969909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:58.969909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:58.969909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:58.969909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:58.971028 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:58.971028 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:58.971028 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:58.971028 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:58.971028 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:58.971028 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:58.971028 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:58.971028 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:58.976600 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 416/825] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:58.977448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46e8f7-0d48-48f0-aea4-2901d3f1d2a1 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:58.979992 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dc679-029c-44f4-b0a3-6958eb395c3f None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:59.016905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:59.017885 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa485e5-f4e1-486c-bc96-ecbbe4e372f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:59.018923 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa485e5-f4e1-486c-bc96-ecbbe4e372f2 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:59.019432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa485e5-f4e1-486c-bc96-ecbbe4e372f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:59.019939 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa485e5-f4e1-486c-bc96-ecbbe4e372f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:59.031677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa485e5-f4e1-486c-bc96-ecbbe4e372f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:59.033563 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa485e5-f4e1-486c-bc96-ecbbe4e372f2 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:59.053453 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-eaa485e5-f4e1-486c-bc96-ecbbe4e372f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:59.053453 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:59.053453 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:59.053453 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:59.053453 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:59.053453 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:59.053453 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:59.053453 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:59.053453 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:59.053453 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:59.053453 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:59.053453 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:59.053453 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:27:59.053453 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:27:59.053453 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:27:59.054524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:27:59.054524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:59.054524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:59.054524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:59.054524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:59.054524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:59.054524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:59.054524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:59.054524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:59.054524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:59.054524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:59.054524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:59.054524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:59.054524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:59.054524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:59.054524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:59.054524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:59.055417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:27:59.055417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:27:59.055417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:27:59.055417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:59.061986 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 410/826] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:27:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:27:59.063450 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa485e5-f4e1-486c-bc96-ecbbe4e372f2 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:59.093087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:59.098083 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dc679-029c-44f4-b0a3-6958eb395c3f None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:59.135860 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae527065-bb19-4409-ad01-d89adbe54e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:59.136522 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae527065-bb19-4409-ad01-d89adbe54e18 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:59.136837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae527065-bb19-4409-ad01-d89adbe54e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:59.137115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae527065-bb19-4409-ad01-d89adbe54e18 None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:59.137911 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae527065-bb19-4409-ad01-d89adbe54e18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:59.138219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae527065-bb19-4409-ad01-d89adbe54e18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:59.139638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae527065-bb19-4409-ad01-d89adbe54e18 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:59.150953 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 417/827] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:59.152202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c0eba-50e1-4438-99bb-45326ae49475 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:59.152494 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c0eba-50e1-4438-99bb-45326ae49475 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:59.152812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c0eba-50e1-4438-99bb-45326ae49475 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:59.157997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 418/828] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:59.159162 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518e6b3-0356-43df-bad2-719eb8aa2086 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:59.159451 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518e6b3-0356-43df-bad2-719eb8aa2086 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:59.159731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518e6b3-0356-43df-bad2-719eb8aa2086 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:59.160980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 419/829] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:27:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:59.161699 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518e6b3-0356-43df-bad2-719eb8aa2086 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:59.174275 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa485e5-f4e1-486c-bc96-ecbbe4e372f2 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:59.209779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77124cef-216f-4843-b6cb-08e9ef7dc08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:59.210445 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77124cef-216f-4843-b6cb-08e9ef7dc08b None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:59.210736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77124cef-216f-4843-b6cb-08e9ef7dc08b None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:59.211014 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77124cef-216f-4843-b6cb-08e9ef7dc08b None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:59.211737 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77124cef-216f-4843-b6cb-08e9ef7dc08b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:59.212069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77124cef-216f-4843-b6cb-08e9ef7dc08b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:59.213452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77124cef-216f-4843-b6cb-08e9ef7dc08b None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:59.225006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 411/830] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:27:59.226379 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27383c02-9626-4a58-91bc-2d512d719600 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:59.226683 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27383c02-9626-4a58-91bc-2d512d719600 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:59.226969 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27383c02-9626-4a58-91bc-2d512d719600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:59.243625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-27383c02-9626-4a58-91bc-2d512d719600 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:59.244861 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-27383c02-9626-4a58-91bc-2d512d719600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:59.245014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:59.263546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518e6b3-0356-43df-bad2-719eb8aa2086 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:59.299952 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a3694-2bf2-4867-bc2b-2461202057dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:59.300632 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a3694-2bf2-4867-bc2b-2461202057dd None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:59.300922 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a3694-2bf2-4867-bc2b-2461202057dd None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:59.301199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a3694-2bf2-4867-bc2b-2461202057dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:59.302030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a3694-2bf2-4867-bc2b-2461202057dd None admin] RBAC: Adding query filter params (role.id=6ff7708b59984b9287cc81d46a57b4b4, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=159cc8befc3a4d57bbd313832084db62) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:59.302951 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a3694-2bf2-4867-bc2b-2461202057dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:59.304376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a3694-2bf2-4867-bc2b-2461202057dd None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:59.310193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 420/831] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:27:59 2022] GET /identity/v3/role_assignments?user.id=159cc8befc3a4d57bbd313832084db62&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=6ff7708b59984b9287cc81d46a57b4b4 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:59.311374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec1175-d9d9-4869-a406-e07d6938e029 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:59.311665 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec1175-d9d9-4869-a406-e07d6938e029 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:59.311946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec1175-d9d9-4869-a406-e07d6938e029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:59.320838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 412/832] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:27:59.321664 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27383c02-9626-4a58-91bc-2d512d719600 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:59.334291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-6eec1175-d9d9-4869-a406-e07d6938e029 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:59.335508 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6eec1175-d9d9-4869-a406-e07d6938e029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:59.355962 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:59.419396 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 421/833] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:27:59.420270 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec1175-d9d9-4869-a406-e07d6938e029 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:59.461336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27383c02-9626-4a58-91bc-2d512d719600 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:59.503242 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276dc228-dc29-4330-915a-c2d16e31e287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:59.504049 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276dc228-dc29-4330-915a-c2d16e31e287 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:59.504340 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276dc228-dc29-4330-915a-c2d16e31e287 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:59.504653 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276dc228-dc29-4330-915a-c2d16e31e287 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:59.513808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:59.514068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276dc228-dc29-4330-915a-c2d16e31e287 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:59.515555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276dc228-dc29-4330-915a-c2d16e31e287 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:59.532084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-276dc228-dc29-4330-915a-c2d16e31e287 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:59.532084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:59.532084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:59.532084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:59.532084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:59.532084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:59.532084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:59.532084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:59.532084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:59.532084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:59.532084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:59.532084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:59.532084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:27:59.532084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:27:59.532084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:27:59.533441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:27:59.533441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:59.533441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:59.533441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:59.533441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:59.533441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:59.533441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:59.533441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:59.533441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:59.533441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:59.533441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:59.533441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:59.533441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:59.533441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:59.533441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:59.533441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:59.533441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:59.534959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:27:59.534959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:27:59.534959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:27:59.534959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:27:59.534959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:27:59.534959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:27:59.534959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:27:59.534959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:59.534959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 413/834] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:27:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:59.534959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276dc228-dc29-4330-915a-c2d16e31e287 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:59.542373 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec1175-d9d9-4869-a406-e07d6938e029 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:59.579245 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c00613-612c-43fd-be34-e3fd25410573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:59.580049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c00613-612c-43fd-be34-e3fd25410573 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:59.580335 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c00613-612c-43fd-be34-e3fd25410573 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:59.580661 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c00613-612c-43fd-be34-e3fd25410573 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:59.581298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c00613-612c-43fd-be34-e3fd25410573 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:59.582663 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c00613-612c-43fd-be34-e3fd25410573 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:59.595116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-31c00613-612c-43fd-be34-e3fd25410573 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 22:27:59.595116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:59.595116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:27:59.595116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:27:59.595116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:27:59.595116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:27:59.595116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:27:59.595116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:27:59.595116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 22:27:59.595116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:59.595116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:27:59.595116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:59.595116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:59.595116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:59.595116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:59.595116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:59.595116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:59.596146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:59.596146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:59.596146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:59.596146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:59.596146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:59.596146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:59.596146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:27:59.596146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:27:59.596146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:27:59.596146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:27:59.596146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:59.596146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:59.596146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:59.596146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:59.596146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:59.596146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:59.597024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:59.597024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:59.597024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:59.597024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:59.597024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:59.597024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:59.597024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:59.597024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:59.597024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:59.597024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:59.597024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:27:59.597024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:27:59.597024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 22:27:59.597024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:27:59.597024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 422/835] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:27:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:59.597833 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c00613-612c-43fd-be34-e3fd25410573 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:59.629253 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:59.654519 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276dc228-dc29-4330-915a-c2d16e31e287 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:59.690111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23c54d-600f-4f27-8c4a-3f145c93ee95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:59.690904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23c54d-600f-4f27-8c4a-3f145c93ee95 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:59.691188 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23c54d-600f-4f27-8c4a-3f145c93ee95 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:59.691460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23c54d-600f-4f27-8c4a-3f145c93ee95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:59.702155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23c54d-600f-4f27-8c4a-3f145c93ee95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:27:59.706775 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c00613-612c-43fd-be34-e3fd25410573 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:59.710317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-1f23c54d-600f-4f27-8c4a-3f145c93ee95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:27:59.710317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:27:59.710317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:27:59.710317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:27:59.710317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:27:59.710317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:27:59.710317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:27:59.710317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:27:59.710317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:27:59.710317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:27:59.710317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:27:59.710317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:27:59.710317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:27:59.710317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:27:59.710317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:27:59.710317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:27:59.711231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:27:59.711231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:27:59.711231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:27:59.711231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:27:59.711231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:27:59.711231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:27:59.711231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:27:59.711231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:27:59.711231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:27:59.711231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:27:59.711231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:27:59.711231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:27:59.711231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:27:59.711231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:27:59.711231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:27:59.711231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:27:59.712075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:27:59.712075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:27:59.712075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:27:59.712075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:27:59.712075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:27:59.712075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:27:59.712075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:27:59.712075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:27:59.712075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:27:59.712075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:27:59.712075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:27:59.712075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:27:59.712075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 414/836] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:27:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:27:59.712795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23c54d-600f-4f27-8c4a-3f145c93ee95 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:59.746306 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5dca2-4eb1-4077-a15a-3d49c9aeef30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:59.747370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5dca2-4eb1-4077-a15a-3d49c9aeef30 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:59.747835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5dca2-4eb1-4077-a15a-3d49c9aeef30 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:59.748309 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5dca2-4eb1-4077-a15a-3d49c9aeef30 None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:59.749318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5dca2-4eb1-4077-a15a-3d49c9aeef30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:59.749936 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5dca2-4eb1-4077-a15a-3d49c9aeef30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:59.751662 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5dca2-4eb1-4077-a15a-3d49c9aeef30 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:59.761675 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:59.765603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 423/837] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:27:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:59.766553 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5dca2-4eb1-4077-a15a-3d49c9aeef30 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:59.824916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23c54d-600f-4f27-8c4a-3f145c93ee95 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:59.859559 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300dc2f0-ece1-4c22-bfe6-4e7c1f355cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:59.860213 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dc2f0-ece1-4c22-bfe6-4e7c1f355cae None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:59.860500 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dc2f0-ece1-4c22-bfe6-4e7c1f355cae None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:59.860817 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dc2f0-ece1-4c22-bfe6-4e7c1f355cae None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:59.861645 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300dc2f0-ece1-4c22-bfe6-4e7c1f355cae None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:59.861959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300dc2f0-ece1-4c22-bfe6-4e7c1f355cae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:59.863462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300dc2f0-ece1-4c22-bfe6-4e7c1f355cae None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:59.866281 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:59.874781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 415/838] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:27:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:59.874781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300dc2f0-ece1-4c22-bfe6-4e7c1f355cae None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:59.881055 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5dca2-4eb1-4077-a15a-3d49c9aeef30 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:59.911279 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743118e6-45ce-4843-a375-e87de5868b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:59.912227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743118e6-45ce-4843-a375-e87de5868b10 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:59.912648 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743118e6-45ce-4843-a375-e87de5868b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:59.912944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743118e6-45ce-4843-a375-e87de5868b10 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:59.913681 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743118e6-45ce-4843-a375-e87de5868b10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:27:59.914018 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743118e6-45ce-4843-a375-e87de5868b10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:59.915351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743118e6-45ce-4843-a375-e87de5868b10 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:59.928617 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 424/839] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:27:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:27:59.928617 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743118e6-45ce-4843-a375-e87de5868b10 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:59.988521 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300dc2f0-ece1-4c22-bfe6-4e7c1f355cae None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:59.991981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:00.025473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fd4da-f522-4ae5-be05-ef09c15c711f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:00.026358 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fd4da-f522-4ae5-be05-ef09c15c711f None admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:00.026642 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fd4da-f522-4ae5-be05-ef09c15c711f None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:00.026966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fd4da-f522-4ae5-be05-ef09c15c711f None admin] PATH_INFO: `/v3/projects/14265887fea245a5a37abbb3a116e352/users/0a7c204a298c42f0bbc32fa9a6ce1f38/roles/698ff821321a43d494fe8940cdd08564` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:00.030684 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743118e6-45ce-4843-a375-e87de5868b10 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:00.042228 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16fd4da-f522-4ae5-be05-ef09c15c711f None admin] RBAC: Authorizing `identity:create_grant(project_id=14265887fea245a5a37abbb3a116e352, user_id=0a7c204a298c42f0bbc32fa9a6ce1f38, role_id=698ff821321a43d494fe8940cdd08564)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:00.043946 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16fd4da-f522-4ae5-be05-ef09c15c711f None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:00.068477 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9150d-9205-49b5-8e27-02600f97d88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:00.069264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9150d-9205-49b5-8e27-02600f97d88a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:00.069561 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9150d-9205-49b5-8e27-02600f97d88a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:00.069837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9150d-9205-49b5-8e27-02600f97d88a None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:00.070535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9150d-9205-49b5-8e27-02600f97d88a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:00.070841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9150d-9205-49b5-8e27-02600f97d88a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:00.072196 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9150d-9205-49b5-8e27-02600f97d88a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:00.081937 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 425/840] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:27:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:00.081937 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9150d-9205-49b5-8e27-02600f97d88a None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:00.111058 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 416/841] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:27:59 2022] PUT /identity/v3/projects/14265887fea245a5a37abbb3a116e352/users/0a7c204a298c42f0bbc32fa9a6ce1f38/roles/698ff821321a43d494fe8940cdd08564 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:28:00.202046 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9150d-9205-49b5-8e27-02600f97d88a None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:00.234915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:00.245722 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f3092-1a3f-4678-9ef0-beebd20db3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:00.246551 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f3092-1a3f-4678-9ef0-beebd20db3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:00.246878 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f3092-1a3f-4678-9ef0-beebd20db3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:00.247165 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f3092-1a3f-4678-9ef0-beebd20db3e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:00.260643 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f3092-1a3f-4678-9ef0-beebd20db3e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:00.262182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f3092-1a3f-4678-9ef0-beebd20db3e8 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:00.282520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-605f3092-1a3f-4678-9ef0-beebd20db3e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:00.282520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:00.282520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:00.282520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:00.282520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:00.282520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:00.282520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:00.282520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:00.282520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:00.282520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:00.282520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:00.282520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:00.282520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:28:00.282520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:28:00.282520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:28:00.283465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:28:00.283465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:00.283465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:00.283465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:00.283465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:00.283465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:00.283465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:00.283465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:00.283465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:00.283465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:00.283465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:00.283465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:00.283465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:00.283465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:00.283465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:00.283465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:00.283465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:00.284437 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:28:00.284437 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:28:00.284437 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:00.284437 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:00.284708 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 426/842] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:00.293063 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fd4da-f522-4ae5-be05-ef09c15c711f None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:00.347160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:00.406127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fd4da-f522-4ae5-be05-ef09c15c711f None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:00.441757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482672b0-2031-42e4-9994-441903af7dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:00.442435 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482672b0-2031-42e4-9994-441903af7dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:00.442728 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482672b0-2031-42e4-9994-441903af7dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:00.443217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482672b0-2031-42e4-9994-441903af7dd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:00.444126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482672b0-2031-42e4-9994-441903af7dd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:00.444448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482672b0-2031-42e4-9994-441903af7dd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:00.445886 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482672b0-2031-42e4-9994-441903af7dd1 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:00.459425 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 417/843] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:00.464469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f3092-1a3f-4678-9ef0-beebd20db3e8 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:00.571571 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f3092-1a3f-4678-9ef0-beebd20db3e8 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:00.606296 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8780d-87d2-4bad-b3f3-d5157ca02b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:00.607049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8780d-87d2-4bad-b3f3-d5157ca02b85 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:00.607290 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8780d-87d2-4bad-b3f3-d5157ca02b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:00.607565 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8780d-87d2-4bad-b3f3-d5157ca02b85 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:00.615617 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8780d-87d2-4bad-b3f3-d5157ca02b85 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:00.617095 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8780d-87d2-4bad-b3f3-d5157ca02b85 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:00.631366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-65d8780d-87d2-4bad-b3f3-d5157ca02b85 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:28:00.631366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:00.631366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:00.631366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:00.631366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:00.631366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:00.631366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:00.631366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:00.631366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:00.631366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:00.631366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:00.631366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:00.631366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:28:00.631366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:28:00.631366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:28:00.631366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:28:00.632313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:00.632313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:00.632313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:28:00.632313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:28:00.632313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:28:00.632313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:28:00.632313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:00.632313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:00.632313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:00.632313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:00.632313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:00.632313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:00.632313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:00.632313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:00.632313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:00.632313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:00.632313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:00.632313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:00.633338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:00.633338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:00.633338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:28:00.633338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:28:00.633338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:28:00.633338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:28:00.633338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:28:00.633338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:28:00.633338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:28:00.633338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:00.636915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 427/844] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:28:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:00.638487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482672b0-2031-42e4-9994-441903af7dd1 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:00.744120 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482672b0-2031-42e4-9994-441903af7dd1 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:00.755783 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:00.780378 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0e05a-8fe2-4c78-958b-b80b8acc23d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:00.781353 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0e05a-8fe2-4c78-958b-b80b8acc23d8 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:00.781888 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0e05a-8fe2-4c78-958b-b80b8acc23d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:00.782344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0e05a-8fe2-4c78-958b-b80b8acc23d8 None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:00.783376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b0e05a-8fe2-4c78-958b-b80b8acc23d8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:00.783907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b0e05a-8fe2-4c78-958b-b80b8acc23d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:00.785534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b0e05a-8fe2-4c78-958b-b80b8acc23d8 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:00.807956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 418/845] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 23 22:28:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:00.813807 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8780d-87d2-4bad-b3f3-d5157ca02b85 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:00.861086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:00.921259 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8780d-87d2-4bad-b3f3-d5157ca02b85 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:00.956561 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0977a5-f613-4d5e-8d5d-8c4763b88cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:00.957241 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0977a5-f613-4d5e-8d5d-8c4763b88cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:00.957526 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0977a5-f613-4d5e-8d5d-8c4763b88cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:00.957802 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0977a5-f613-4d5e-8d5d-8c4763b88cd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:01.028232 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0977a5-f613-4d5e-8d5d-8c4763b88cd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:01.029752 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0977a5-f613-4d5e-8d5d-8c4763b88cd4 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:01.069642 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:01.132310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 428/846] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:28:00 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:28:01.137758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0e05a-8fe2-4c78-958b-b80b8acc23d8 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:01.242461 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0e05a-8fe2-4c78-958b-b80b8acc23d8 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:01.296272 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863c517-8b30-4289-a7ec-ae39e5b51c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:01.297124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863c517-8b30-4289-a7ec-ae39e5b51c8a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:01.297408 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863c517-8b30-4289-a7ec-ae39e5b51c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:01.297682 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863c517-8b30-4289-a7ec-ae39e5b51c8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:01.310309 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863c517-8b30-4289-a7ec-ae39e5b51c8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:01.311804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863c517-8b30-4289-a7ec-ae39e5b51c8a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:01.329691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-e863c517-8b30-4289-a7ec-ae39e5b51c8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:01.329691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:01.329691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:01.329691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:01.329691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:01.329691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:01.329691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:01.329691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:01.329691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:01.329691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:01.329691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:01.329691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:01.329691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:28:01.329691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:28:01.329691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:28:01.335156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:28:01.335156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:01.335156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:01.335156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:01.335156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:01.335156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:01.335156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:01.335156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:01.335156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:01.335156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:01.335156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:01.335156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:01.335156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:01.335156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:01.335156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:01.335156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:01.335156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:01.336599 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:28:01.336599 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:28:01.336599 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:01.336599 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:01.336599 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 419/847] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:01.337636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0977a5-f613-4d5e-8d5d-8c4763b88cd4 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:01.444150 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0977a5-f613-4d5e-8d5d-8c4763b88cd4 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:01.478839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabfb112-d614-47c5-82e6-8bf6325e3f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:01.479503 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfb112-d614-47c5-82e6-8bf6325e3f51 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:01.479788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfb112-d614-47c5-82e6-8bf6325e3f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:01.480068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfb112-d614-47c5-82e6-8bf6325e3f51 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:01.480817 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabfb112-d614-47c5-82e6-8bf6325e3f51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:01.481128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabfb112-d614-47c5-82e6-8bf6325e3f51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:01.482485 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabfb112-d614-47c5-82e6-8bf6325e3f51 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:01.494090 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 429/848] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:01.496905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:01.499594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863c517-8b30-4289-a7ec-ae39e5b51c8a None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:01.524558 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0cab0-c473-45c8-b691-3cea986c28d4 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:01.524887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0cab0-c473-45c8-b691-3cea986c28d4 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:01.525182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0cab0-c473-45c8-b691-3cea986c28d4 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:01.526816 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 430/849] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:28:01.527561 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:01.533867 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619a054-5756-4388-8771-531adc1828a8 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:01.534160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619a054-5756-4388-8771-531adc1828a8 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:01.534446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619a054-5756-4388-8771-531adc1828a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:01.551542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-4619a054-5756-4388-8771-531adc1828a8 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:01.553362 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-4619a054-5756-4388-8771-531adc1828a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:01.624603 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863c517-8b30-4289-a7ec-ae39e5b51c8a None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:01.631655 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 431/850] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:01.655297 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619a054-5756-4388-8771-531adc1828a8 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:01.669355 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ffa5d-c9a2-4571-bb01-454da794acfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:01.670412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ffa5d-c9a2-4571-bb01-454da794acfc None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:01.670708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ffa5d-c9a2-4571-bb01-454da794acfc None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:01.670995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ffa5d-c9a2-4571-bb01-454da794acfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:01.680792 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ffa5d-c9a2-4571-bb01-454da794acfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:01.682943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ffa5d-c9a2-4571-bb01-454da794acfc None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:01.699474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-a59ffa5d-c9a2-4571-bb01-454da794acfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:28:01.699474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:01.699474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:01.699474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:01.699474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:01.699474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:01.699474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:01.699474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:01.699474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:01.699474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:01.699474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:01.699474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:01.699474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:28:01.699474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:28:01.699474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:28:01.700350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:28:01.700350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:01.700350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:01.700350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:01.700350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:01.700350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:01.700350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:01.700350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:01.700350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:01.700350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:01.700350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:01.700350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:01.700350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:01.700350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:01.700350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:01.700350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:01.700350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:01.701277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:28:01.701277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:28:01.701277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:28:01.701277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:28:01.701277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:28:01.701277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:28:01.701277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:28:01.701277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:01.701277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 420/851] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:28:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:01.708284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ffa5d-c9a2-4571-bb01-454da794acfc None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:01.781857 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619a054-5756-4388-8771-531adc1828a8 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:01.815378 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:01.822544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:01.831813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f1f83-600a-4af2-84ae-df70dced585a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:01.832797 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f1f83-600a-4af2-84ae-df70dced585a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:01.833094 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f1f83-600a-4af2-84ae-df70dced585a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:01.833400 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f1f83-600a-4af2-84ae-df70dced585a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:01.834349 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f1f83-600a-4af2-84ae-df70dced585a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:01.836038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f1f83-600a-4af2-84ae-df70dced585a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:01.841147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ffa5d-c9a2-4571-bb01-454da794acfc None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:01.851213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-7b0f1f83-600a-4af2-84ae-df70dced585a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 22:28:01.851213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:01.851213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:28:01.851213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:28:01.851213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:28:01.851213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:28:01.851213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:28:01.851213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:01.851213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 22:28:01.851213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:01.851213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:28:01.851213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:01.851213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:01.851213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:01.851213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:01.851213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:01.852425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:01.852425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:01.852425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:01.852425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:01.852425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:01.852425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:01.852425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:01.852425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:28:01.852425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:28:01.852425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:28:01.852425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:28:01.852425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:01.852425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:01.852425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:01.852425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:01.852425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:01.853500 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:01.853500 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:01.853500 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:01.853500 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:01.853500 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:01.853500 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:01.853500 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:01.853500 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:01.853500 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:01.853500 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:01.853500 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:01.853500 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:28:01.853500 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:01.853500 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 22:28:01.853500 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:01.856723 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 432/852] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Jun 23 22:28:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:01.859188 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f1f83-600a-4af2-84ae-df70dced585a None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:01.894708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84081bca-4c92-4891-b46e-c62b98a056cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:01.896836 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84081bca-4c92-4891-b46e-c62b98a056cf None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:01.897182 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84081bca-4c92-4891-b46e-c62b98a056cf None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:01.897514 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84081bca-4c92-4891-b46e-c62b98a056cf None admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:01.899034 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84081bca-4c92-4891-b46e-c62b98a056cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:01.899355 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84081bca-4c92-4891-b46e-c62b98a056cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:01.900846 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84081bca-4c92-4891-b46e-c62b98a056cf None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:01.914438 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 421/853] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:28:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:01.920091 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84081bca-4c92-4891-b46e-c62b98a056cf None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:01.978355 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f1f83-600a-4af2-84ae-df70dced585a None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:02.014604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df0ad9-c738-4f91-b5a9-69e9daa46207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:02.015647 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df0ad9-c738-4f91-b5a9-69e9daa46207 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:02.016164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df0ad9-c738-4f91-b5a9-69e9daa46207 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:02.016755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df0ad9-c738-4f91-b5a9-69e9daa46207 None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:02.018286 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df0ad9-c738-4f91-b5a9-69e9daa46207 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:02.019071 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df0ad9-c738-4f91-b5a9-69e9daa46207 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:02.021923 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df0ad9-c738-4f91-b5a9-69e9daa46207 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:02.027438 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 433/854] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 22:28:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:02.031417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84081bca-4c92-4891-b46e-c62b98a056cf None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:02.032477 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df0ad9-c738-4f91-b5a9-69e9daa46207 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:02.068832 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafef94-6f08-4c83-b8ff-540d3d101f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:02.069768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafef94-6f08-4c83-b8ff-540d3d101f90 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:02.069883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafef94-6f08-4c83-b8ff-540d3d101f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:02.070167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafef94-6f08-4c83-b8ff-540d3d101f90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:02.070997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdafef94-6f08-4c83-b8ff-540d3d101f90 None admin] RBAC: Adding query filter params (role.id=6ff7708b59984b9287cc81d46a57b4b4, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=b321586cdb3d4d25904472aaa4adba71) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:02.071324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdafef94-6f08-4c83-b8ff-540d3d101f90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:02.072925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdafef94-6f08-4c83-b8ff-540d3d101f90 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:02.084891 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 422/855] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:28:01 2022] GET /identity/v3/role_assignments?user.id=b321586cdb3d4d25904472aaa4adba71&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=6ff7708b59984b9287cc81d46a57b4b4 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:02.107882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:02.160070 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df0ad9-c738-4f91-b5a9-69e9daa46207 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:02.200023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea2e41-9371-4a00-871c-5c98a8c449a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:02.200907 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea2e41-9371-4a00-871c-5c98a8c449a9 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:02.201302 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea2e41-9371-4a00-871c-5c98a8c449a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:02.201688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea2e41-9371-4a00-871c-5c98a8c449a9 None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:02.202530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea2e41-9371-4a00-871c-5c98a8c449a9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:02.202952 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea2e41-9371-4a00-871c-5c98a8c449a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:02.204474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea2e41-9371-4a00-871c-5c98a8c449a9 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:02.211442 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 434/856] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 22:28:02 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:02.288979 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba4c5d-96e0-4435-89f9-6c39b4f5bc30 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:02.289408 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba4c5d-96e0-4435-89f9-6c39b4f5bc30 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:02.289788 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba4c5d-96e0-4435-89f9-6c39b4f5bc30 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:02.291755 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 423/857] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:02.297493 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f25a9-1697-4838-aaac-091ff323fec8 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:02.297789 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f25a9-1697-4838-aaac-091ff323fec8 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:02.298074 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f25a9-1697-4838-aaac-091ff323fec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:02.307813 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0107db-ba5d-41ae-bfd5-cc379a15751b None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:02.308104 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0107db-ba5d-41ae-bfd5-cc379a15751b None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:02.308388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0107db-ba5d-41ae-bfd5-cc379a15751b None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:02.309951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 424/858] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:02.315353 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16569a25-52f3-4ae2-987c-e0c5c010ac6a None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:02.315650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16569a25-52f3-4ae2-987c-e0c5c010ac6a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:02.315938 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16569a25-52f3-4ae2-987c-e0c5c010ac6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:02.321256 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-4c5f25a9-1697-4838-aaac-091ff323fec8 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:02.322609 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-4c5f25a9-1697-4838-aaac-091ff323fec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:02.323233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:02.335907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-16569a25-52f3-4ae2-987c-e0c5c010ac6a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:02.337197 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-16569a25-52f3-4ae2-987c-e0c5c010ac6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:02.404546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 435/859] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:02.422223 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 425/860] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:28:02.434724 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f25a9-1697-4838-aaac-091ff323fec8 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:02.447118 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16569a25-52f3-4ae2-987c-e0c5c010ac6a None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:02.499095 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:02.560735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16569a25-52f3-4ae2-987c-e0c5c010ac6a None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:02.569098 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f25a9-1697-4838-aaac-091ff323fec8 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:02.596284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad58c72-8d91-4c81-8d6f-f1948c80da4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:02.597404 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad58c72-8d91-4c81-8d6f-f1948c80da4c None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:02.597894 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad58c72-8d91-4c81-8d6f-f1948c80da4c None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:02.598363 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad58c72-8d91-4c81-8d6f-f1948c80da4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:02.601144 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:02.608886 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db5491-2a70-45c0-b20e-f48bc6c7b28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:02.609565 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db5491-2a70-45c0-b20e-f48bc6c7b28b None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:02.609856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db5491-2a70-45c0-b20e-f48bc6c7b28b None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:02.610141 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db5491-2a70-45c0-b20e-f48bc6c7b28b None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:02.610975 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db5491-2a70-45c0-b20e-f48bc6c7b28b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:02.611929 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad58c72-8d91-4c81-8d6f-f1948c80da4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:28:02.612331 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db5491-2a70-45c0-b20e-f48bc6c7b28b None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:02.620906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-cad58c72-8d91-4c81-8d6f-f1948c80da4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:28:02.620906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:02.620906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:02.620906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:02.620906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:02.620906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:02.620906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:02.620906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:02.620906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:02.620906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:02.620906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:02.620906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:02.620906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:28:02.620906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:28:02.620906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:28:02.620906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:28:02.621866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:28:02.621866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:28:02.621866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:28:02.621866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:28:02.621866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:02.621866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:02.621866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:02.621866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:02.621866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:02.621866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:02.621866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:02.621866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:02.621866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:02.621866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:02.621866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:02.621866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:02.622692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:02.622692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:02.622692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:02.622692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:02.622692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:28:02.622692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:28:02.622692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:28:02.622692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:28:02.622692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:28:02.622692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:28:02.622692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:28:02.622692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:02.625286 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 426/861] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:28:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:02.631469 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad58c72-8d91-4c81-8d6f-f1948c80da4c None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:02.708818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 436/862] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 22:28:02 2022] POST /identity/v3/services => generated 251 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:28:02.751218 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad58c72-8d91-4c81-8d6f-f1948c80da4c None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:02.796742 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538c2da-f198-4ef0-a5fe-58586bbc3cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:02.797435 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538c2da-f198-4ef0-a5fe-58586bbc3cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:02.797715 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538c2da-f198-4ef0-a5fe-58586bbc3cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:02.797991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538c2da-f198-4ef0-a5fe-58586bbc3cc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:02.798753 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538c2da-f198-4ef0-a5fe-58586bbc3cc2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:02.799065 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538c2da-f198-4ef0-a5fe-58586bbc3cc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:02.800459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538c2da-f198-4ef0-a5fe-58586bbc3cc2 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:02.810680 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 427/863] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:28:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:02.817091 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db5491-2a70-45c0-b20e-f48bc6c7b28b None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:02.934087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db5491-2a70-45c0-b20e-f48bc6c7b28b None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:02.946079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:02.970981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb8053-6b02-494c-ae4c-56057a65a9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:02.971846 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb8053-6b02-494c-ae4c-56057a65a9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:02.972136 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb8053-6b02-494c-ae4c-56057a65a9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:02.972465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb8053-6b02-494c-ae4c-56057a65a9c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:02.985314 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb8053-6b02-494c-ae4c-56057a65a9c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:02.986876 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb8053-6b02-494c-ae4c-56057a65a9c0 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:03.005326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-19cb8053-6b02-494c-ae4c-56057a65a9c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:03.005326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:03.005326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:03.005326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:03.005326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:03.005326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:03.005326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:03.005326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:03.005326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:03.005326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:03.005326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:03.005326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:03.005326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:28:03.005326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:28:03.005326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:28:03.006227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:28:03.006227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:03.006227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:03.006227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:03.006227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:03.006227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:03.006227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:03.006227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:03.006227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:03.006227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:03.006227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:03.006227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:03.006227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:03.006227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:03.006227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:03.006227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:03.006227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:03.007651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:28:03.007651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:28:03.007651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:03.007651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:03.007651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 437/864] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:03.012707 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538c2da-f198-4ef0-a5fe-58586bbc3cc2 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:03.070594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:03.122492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538c2da-f198-4ef0-a5fe-58586bbc3cc2 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:03.162676 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177dbc6-2cdd-4bb5-98c4-7b69aa157470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:03.163720 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177dbc6-2cdd-4bb5-98c4-7b69aa157470 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:03.164206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177dbc6-2cdd-4bb5-98c4-7b69aa157470 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:03.164737 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177dbc6-2cdd-4bb5-98c4-7b69aa157470 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:03.165926 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177dbc6-2cdd-4bb5-98c4-7b69aa157470 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:03.166445 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177dbc6-2cdd-4bb5-98c4-7b69aa157470 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:03.167999 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177dbc6-2cdd-4bb5-98c4-7b69aa157470 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:03.180351 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 428/865] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:03.185577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb8053-6b02-494c-ae4c-56057a65a9c0 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:03.295958 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb8053-6b02-494c-ae4c-56057a65a9c0 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:03.335508 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e0b6f-21f3-46bd-a8bc-a831b4615c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:03.335508 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e0b6f-21f3-46bd-a8bc-a831b4615c7e None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:03.335508 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e0b6f-21f3-46bd-a8bc-a831b4615c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:03.335508 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e0b6f-21f3-46bd-a8bc-a831b4615c7e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:03.343029 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e0b6f-21f3-46bd-a8bc-a831b4615c7e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:03.344454 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e0b6f-21f3-46bd-a8bc-a831b4615c7e None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:03.359474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-212e0b6f-21f3-46bd-a8bc-a831b4615c7e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:28:03.359474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:03.359474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:03.359474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:03.359474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:03.359474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:03.359474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:03.359474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:03.359474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:03.359474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:03.359474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:03.359474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:03.359474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:28:03.359474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:28:03.359474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:28:03.359474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:28:03.360491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:03.360491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:03.360491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:28:03.360491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:28:03.360491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:28:03.360491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:28:03.360491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:03.360491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:03.360491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:03.360491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:03.360491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:03.360491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:03.360491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:03.360491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:03.360491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:03.360491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:03.360491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:03.360491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:03.365630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:03.365630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:03.365630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:28:03.365630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:28:03.365630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:28:03.365630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:28:03.365630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:28:03.365630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:28:03.365630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:28:03.365630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:03.365630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 438/866] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 23 22:28:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:03.367700 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177dbc6-2cdd-4bb5-98c4-7b69aa157470 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:03.474619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177dbc6-2cdd-4bb5-98c4-7b69aa157470 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:03.492939 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:03.509821 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fc90c-712b-4c31-9c86-d55bdbb6dbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:03.510766 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fc90c-712b-4c31-9c86-d55bdbb6dbba None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:03.511215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fc90c-712b-4c31-9c86-d55bdbb6dbba None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:03.511636 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fc90c-712b-4c31-9c86-d55bdbb6dbba None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:03.512631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185fc90c-712b-4c31-9c86-d55bdbb6dbba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:03.513104 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185fc90c-712b-4c31-9c86-d55bdbb6dbba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:03.514680 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185fc90c-712b-4c31-9c86-d55bdbb6dbba None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:03.533354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 429/867] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 23 22:28:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:03.541966 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e0b6f-21f3-46bd-a8bc-a831b4615c7e None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:03.569917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:03.648901 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e0b6f-21f3-46bd-a8bc-a831b4615c7e None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:03.684363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390048f-747e-4d38-996f-7ab14773599a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:03.685052 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390048f-747e-4d38-996f-7ab14773599a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:03.685345 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390048f-747e-4d38-996f-7ab14773599a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:03.685630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390048f-747e-4d38-996f-7ab14773599a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:03.756783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390048f-747e-4d38-996f-7ab14773599a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:03.758313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390048f-747e-4d38-996f-7ab14773599a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:03.782940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:03.864192 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 439/868] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:28:03 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:28:03.870297 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fc90c-712b-4c31-9c86-d55bdbb6dbba None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:03.977867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fc90c-712b-4c31-9c86-d55bdbb6dbba None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:04.013628 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075a098-4652-4807-a9de-7063791d6c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:04.014711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075a098-4652-4807-a9de-7063791d6c6c None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:04.015140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075a098-4652-4807-a9de-7063791d6c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:04.015558 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075a098-4652-4807-a9de-7063791d6c6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:04.026778 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075a098-4652-4807-a9de-7063791d6c6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:04.028423 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075a098-4652-4807-a9de-7063791d6c6c None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:04.046992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-0075a098-4652-4807-a9de-7063791d6c6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:04.046992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:04.046992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:04.046992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:04.046992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:04.046992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:04.046992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:04.046992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:04.046992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:04.046992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:04.046992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:04.046992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:04.046992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:28:04.046992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:28:04.046992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:28:04.048068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:28:04.048068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:04.048068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:04.048068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:04.048068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:04.048068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:04.048068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:04.048068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:04.048068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:04.048068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:04.048068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:04.048068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:04.048068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:04.048068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:04.048068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:04.048068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:04.048068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:04.049096 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:28:04.049096 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:28:04.049096 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:04.049096 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:04.054733 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 430/869] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:04.056094 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390048f-747e-4d38-996f-7ab14773599a None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:04.164184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390048f-747e-4d38-996f-7ab14773599a None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:04.187763 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:04.199309 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffa119-8006-417b-8808-3dc360d60b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:04.199975 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa119-8006-417b-8808-3dc360d60b0c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:04.200263 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa119-8006-417b-8808-3dc360d60b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:04.200536 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa119-8006-417b-8808-3dc360d60b0c None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:04.201284 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffa119-8006-417b-8808-3dc360d60b0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:04.201589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffa119-8006-417b-8808-3dc360d60b0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:04.202944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffa119-8006-417b-8808-3dc360d60b0c None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:04.215185 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 440/870] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:28:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:04.221744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075a098-4652-4807-a9de-7063791d6c6c None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:04.247140 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515b39c-699f-4a68-8991-3e8458825736 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:04.247450 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515b39c-699f-4a68-8991-3e8458825736 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:04.247715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515b39c-699f-4a68-8991-3e8458825736 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:04.249362 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 441/871] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:28:04.250620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:04.254384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74349292-f1e7-423b-bbc1-b63764460edd None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:04.254678 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74349292-f1e7-423b-bbc1-b63764460edd None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:04.254965 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74349292-f1e7-423b-bbc1-b63764460edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:04.280397 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-74349292-f1e7-423b-bbc1-b63764460edd None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:04.281949 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-74349292-f1e7-423b-bbc1-b63764460edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:04.359203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 442/872] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:04 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:28:04.360502 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075a098-4652-4807-a9de-7063791d6c6c None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:04.386848 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74349292-f1e7-423b-bbc1-b63764460edd None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:04.407125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc4e1a-46d3-4bbf-94e2-14b6e97151c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:04.408673 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc4e1a-46d3-4bbf-94e2-14b6e97151c8 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:04.409202 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc4e1a-46d3-4bbf-94e2-14b6e97151c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:04.410148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc4e1a-46d3-4bbf-94e2-14b6e97151c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:04.421120 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc4e1a-46d3-4bbf-94e2-14b6e97151c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:04.423091 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc4e1a-46d3-4bbf-94e2-14b6e97151c8 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:04.440951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-0bbc4e1a-46d3-4bbf-94e2-14b6e97151c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:28:04.440951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:04.440951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:04.440951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:04.440951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:04.440951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:04.440951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:04.440951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:04.440951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:04.440951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:04.440951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:04.440951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:04.440951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:28:04.440951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:28:04.440951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:28:04.441882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:28:04.441882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:04.441882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:04.441882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:04.441882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:04.441882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:04.441882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:04.441882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:04.441882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:04.441882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:04.441882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:04.441882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:04.441882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:04.441882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:04.441882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:04.441882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:04.441882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:04.442838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:28:04.442838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:28:04.442838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:28:04.442838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:28:04.442838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:28:04.442838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:28:04.442838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:28:04.442838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:04.445297 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 431/873] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:28:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:04.450859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc4e1a-46d3-4bbf-94e2-14b6e97151c8 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:04.503032 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74349292-f1e7-423b-bbc1-b63764460edd None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:04.517683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:04.545742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4b30c-52d4-467d-bf63-d660bae4908a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:04.546567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4b30c-52d4-467d-bf63-d660bae4908a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:04.547211 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4b30c-52d4-467d-bf63-d660bae4908a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:04.548020 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4b30c-52d4-467d-bf63-d660bae4908a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:04.549097 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4b30c-52d4-467d-bf63-d660bae4908a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:04.550804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:04.551042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4b30c-52d4-467d-bf63-d660bae4908a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:04.564731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-83b4b30c-52d4-467d-bf63-d660bae4908a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:28:04.564731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:04.564731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:28:04.564731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:28:04.564731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:28:04.564731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:28:04.564731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:28:04.564731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:04.564731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:28:04.564731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:04.564731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:28:04.564731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:04.564731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:04.564731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:04.564731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:04.564731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:04.564731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:04.567418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:04.567418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:04.567418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:04.567418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:04.567418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:04.567418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:04.567418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:28:04.567418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:28:04.567418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:28:04.567418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:28:04.567418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:04.567418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:04.567418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:04.567418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:04.567418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:04.567418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:04.568436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:04.568436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:04.568436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:04.568436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:04.568436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:04.568436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:04.568436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:04.568436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:04.568436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:04.568436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:04.568436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:28:04.568436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:04.568436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:28:04.568436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:04.568436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 443/874] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 22:28:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:04.573178 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fcb86-6cae-41f3-ae9d-a282f4f63eef None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:04.573178 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fcb86-6cae-41f3-ae9d-a282f4f63eef None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:04.573178 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fcb86-6cae-41f3-ae9d-a282f4f63eef None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:04.573178 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 444/875] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:04.573178 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01fcb86-6cae-41f3-ae9d-a282f4f63eef None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:04.576818 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc4e1a-46d3-4bbf-94e2-14b6e97151c8 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:04.612950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b27e6-8977-4dfa-8da0-888f6c9ef3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:04.613614 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b27e6-8977-4dfa-8da0-888f6c9ef3ae None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:04.613899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b27e6-8977-4dfa-8da0-888f6c9ef3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:04.614175 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b27e6-8977-4dfa-8da0-888f6c9ef3ae None admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:04.615519 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b27e6-8977-4dfa-8da0-888f6c9ef3ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:04.615828 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b27e6-8977-4dfa-8da0-888f6c9ef3ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:04.617281 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b27e6-8977-4dfa-8da0-888f6c9ef3ae None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:04.628608 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 432/876] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:28:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:04.629823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3e6ea-7f2f-4743-a3c8-ae3284d9e54a None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:04.630887 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3e6ea-7f2f-4743-a3c8-ae3284d9e54a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:04.631175 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3e6ea-7f2f-4743-a3c8-ae3284d9e54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:04.648220 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-e9a3e6ea-7f2f-4743-a3c8-ae3284d9e54a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:04.649445 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e9a3e6ea-7f2f-4743-a3c8-ae3284d9e54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:04.687081 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01fcb86-6cae-41f3-ae9d-a282f4f63eef None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:04.716979 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 433/877] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:04 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:04.721562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3e6ea-7f2f-4743-a3c8-ae3284d9e54a None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:04.721676 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3cbf5-22b7-4c11-848a-3b2e43c8e913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:04.722276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3cbf5-22b7-4c11-848a-3b2e43c8e913 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:04.722589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3cbf5-22b7-4c11-848a-3b2e43c8e913 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:04.722840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3cbf5-22b7-4c11-848a-3b2e43c8e913 None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:04.723541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3cbf5-22b7-4c11-848a-3b2e43c8e913 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:04.724378 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3cbf5-22b7-4c11-848a-3b2e43c8e913 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:04.725769 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3cbf5-22b7-4c11-848a-3b2e43c8e913 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:04.736615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 445/878] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 23 22:28:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:04.737899 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3cbf5-22b7-4c11-848a-3b2e43c8e913 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:04.783833 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:04.810357 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:04.839909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3e6ea-7f2f-4743-a3c8-ae3284d9e54a None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:04.847752 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3cbf5-22b7-4c11-848a-3b2e43c8e913 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:04.876005 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26c705-381b-4ba9-84bc-928c37b8b99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:04.876723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26c705-381b-4ba9-84bc-928c37b8b99e None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:04.877010 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26c705-381b-4ba9-84bc-928c37b8b99e None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:04.877293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26c705-381b-4ba9-84bc-928c37b8b99e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:04.878118 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26c705-381b-4ba9-84bc-928c37b8b99e None admin] RBAC: Adding query filter params (role.id=698ff821321a43d494fe8940cdd08564, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=0a7c204a298c42f0bbc32fa9a6ce1f38) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:04.878432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26c705-381b-4ba9-84bc-928c37b8b99e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:04.879897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26c705-381b-4ba9-84bc-928c37b8b99e None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:04.885615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c060cc-6b8c-480a-8f60-91c02b2eaee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:04.886127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 434/879] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:28:04 2022] GET /identity/v3/role_assignments?user.id=0a7c204a298c42f0bbc32fa9a6ce1f38&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=698ff821321a43d494fe8940cdd08564 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:04.886704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c060cc-6b8c-480a-8f60-91c02b2eaee0 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:04.886902 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26c705-381b-4ba9-84bc-928c37b8b99e None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:04.887366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c060cc-6b8c-480a-8f60-91c02b2eaee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:04.887651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c060cc-6b8c-480a-8f60-91c02b2eaee0 None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:04.888371 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c060cc-6b8c-480a-8f60-91c02b2eaee0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:04.888718 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c060cc-6b8c-480a-8f60-91c02b2eaee0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:04.890133 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c060cc-6b8c-480a-8f60-91c02b2eaee0 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:04.904605 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 446/880] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 23 22:28:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:04.904605 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6ba58-3393-4dfa-8d0d-cd5464dfea4c None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:04.904605 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6ba58-3393-4dfa-8d0d-cd5464dfea4c None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:04.904605 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6ba58-3393-4dfa-8d0d-cd5464dfea4c None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:04.905035 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 447/881] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:04.906470 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdbab4-4dc0-42ac-900d-25732bbb7ba7 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:04.906844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdbab4-4dc0-42ac-900d-25732bbb7ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:04.907192 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdbab4-4dc0-42ac-900d-25732bbb7ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:04.931658 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-d0bdbab4-4dc0-42ac-900d-25732bbb7ba7 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:04.933521 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d0bdbab4-4dc0-42ac-900d-25732bbb7ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:05.020471 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 448/882] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:04 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:05.030883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26c705-381b-4ba9-84bc-928c37b8b99e None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:05.047890 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdbab4-4dc0-42ac-900d-25732bbb7ba7 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:05.078201 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609473ad-3106-4f38-8734-0257d7237db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:05.078766 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609473ad-3106-4f38-8734-0257d7237db0 None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:05.079053 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609473ad-3106-4f38-8734-0257d7237db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:05.079336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609473ad-3106-4f38-8734-0257d7237db0 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:05.080088 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609473ad-3106-4f38-8734-0257d7237db0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:05.081488 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609473ad-3106-4f38-8734-0257d7237db0 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:05.085118 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:05.092565 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:05.162533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdbab4-4dc0-42ac-900d-25732bbb7ba7 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:05.196063 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 435/883] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 22:28:04 2022] POST /identity/v3/services => generated 269 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:28:05.207685 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761c636-e20c-4859-87fe-698bb3d4948d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:05.209143 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761c636-e20c-4859-87fe-698bb3d4948d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:05.209430 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761c636-e20c-4859-87fe-698bb3d4948d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:05.209708 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761c636-e20c-4859-87fe-698bb3d4948d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:05.210376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761c636-e20c-4859-87fe-698bb3d4948d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:05.211870 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761c636-e20c-4859-87fe-698bb3d4948d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:05.226599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-2761c636-e20c-4859-87fe-698bb3d4948d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 22:28:05.226599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:05.226599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:28:05.226599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:28:05.226599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:28:05.226599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:28:05.226599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:28:05.226599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:05.226599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 22:28:05.226599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:05.226599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:28:05.226599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:05.226599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:05.226599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:05.226599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:05.226599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:05.226599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:05.227720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:05.227720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:05.227720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:05.227720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:05.227720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:05.227720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:05.227720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:28:05.227720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:28:05.227720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:28:05.227720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:28:05.227720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:05.227720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:05.227720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:05.227720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:05.227720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:05.227720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:05.228602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:05.228602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:05.228602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:05.228602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:05.228602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:05.228602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:05.228602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:05.228602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:05.228602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:05.228602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:05.228602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:28:05.228602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:05.228602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 22:28:05.228602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:05.232814 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 449/884] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:28:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:05.234545 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609473ad-3106-4f38-8734-0257d7237db0 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:05.352900 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609473ad-3106-4f38-8734-0257d7237db0 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:05.389898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ace7b7-72f9-4a0d-994d-07177b39cef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:05.390571 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ace7b7-72f9-4a0d-994d-07177b39cef3 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:05.390857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ace7b7-72f9-4a0d-994d-07177b39cef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:05.391136 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ace7b7-72f9-4a0d-994d-07177b39cef3 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:05.391848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ace7b7-72f9-4a0d-994d-07177b39cef3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:05.392155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ace7b7-72f9-4a0d-994d-07177b39cef3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:05.393590 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ace7b7-72f9-4a0d-994d-07177b39cef3 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:05.399150 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 436/885] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:28:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:05.404676 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761c636-e20c-4859-87fe-698bb3d4948d None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:05.509565 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:05.516422 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761c636-e20c-4859-87fe-698bb3d4948d None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:05.553988 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fa589-a8b3-47c8-b363-09840dfd54f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:05.554659 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fa589-a8b3-47c8-b363-09840dfd54f0 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:05.554946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fa589-a8b3-47c8-b363-09840dfd54f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:05.555228 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fa589-a8b3-47c8-b363-09840dfd54f0 None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:05.555940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fa589-a8b3-47c8-b363-09840dfd54f0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:05.556247 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fa589-a8b3-47c8-b363-09840dfd54f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:05.557743 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fa589-a8b3-47c8-b363-09840dfd54f0 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:05.563167 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 450/886] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:28:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:05.568891 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ace7b7-72f9-4a0d-994d-07177b39cef3 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:05.632612 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:05.675951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ace7b7-72f9-4a0d-994d-07177b39cef3 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:05.711979 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5aab2-9dbf-4bff-84e3-37e1f9e71a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:05.712728 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5aab2-9dbf-4bff-84e3-37e1f9e71a13 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:05.713014 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5aab2-9dbf-4bff-84e3-37e1f9e71a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:05.713292 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5aab2-9dbf-4bff-84e3-37e1f9e71a13 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:05.714081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5aab2-9dbf-4bff-84e3-37e1f9e71a13 None admin] RBAC: Adding query filter params (interface=public, service_id=d7928c34515b4b148baefd0e720e37aa, region_id=RegionOne) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:05.714396 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5aab2-9dbf-4bff-84e3-37e1f9e71a13 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:05.715771 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5aab2-9dbf-4bff-84e3-37e1f9e71a13 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:05.725828 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 437/887] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 23 22:28:05 2022] GET /identity/v3/endpoints?service_id=d7928c34515b4b148baefd0e720e37aa&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:05.727688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fa589-a8b3-47c8-b363-09840dfd54f0 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:05.832186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fa589-a8b3-47c8-b363-09840dfd54f0 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:05.871720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cd020-2d91-47e4-9a26-523a7d1dfa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:05.872394 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cd020-2d91-47e4-9a26-523a7d1dfa5a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:05.872708 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cd020-2d91-47e4-9a26-523a7d1dfa5a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:05.873071 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cd020-2d91-47e4-9a26-523a7d1dfa5a None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:05.873762 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71cd020-2d91-47e4-9a26-523a7d1dfa5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:05.875126 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71cd020-2d91-47e4-9a26-523a7d1dfa5a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:05.881391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 451/888] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 23 22:28:05 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:28:07.041953 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba115c2a-7f59-4644-8516-80e93e710da7 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:07.042505 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba115c2a-7f59-4644-8516-80e93e710da7 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:07.042505 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba115c2a-7f59-4644-8516-80e93e710da7 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:07.044001 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 438/889] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:07.049554 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f749bc5-6d68-479a-8ae1-79694985ed79 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:07.050370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f749bc5-6d68-479a-8ae1-79694985ed79 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:07.050886 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f749bc5-6d68-479a-8ae1-79694985ed79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:07.070465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-5f749bc5-6d68-479a-8ae1-79694985ed79 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:07.072002 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-5f749bc5-6d68-479a-8ae1-79694985ed79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:07.091146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:07.097682 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edab5b-38f3-4e29-8fb6-19cb7a7e3af3 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:07.097972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edab5b-38f3-4e29-8fb6-19cb7a7e3af3 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:07.098254 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edab5b-38f3-4e29-8fb6-19cb7a7e3af3 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:07.100349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 439/890] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:07.105310 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01ef0d-cfa8-4379-a19f-b911d9e24d1e None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:07.105723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01ef0d-cfa8-4379-a19f-b911d9e24d1e None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:07.106082 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01ef0d-cfa8-4379-a19f-b911d9e24d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:07.125231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-5e01ef0d-cfa8-4379-a19f-b911d9e24d1e None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:07.126756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-5e01ef0d-cfa8-4379-a19f-b911d9e24d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:07.152908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 452/891] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:28:07.177061 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f749bc5-6d68-479a-8ae1-79694985ed79 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:07.216280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 440/892] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:28:07.240640 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01ef0d-cfa8-4379-a19f-b911d9e24d1e None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:07.295251 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f749bc5-6d68-479a-8ae1-79694985ed79 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:07.329059 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:07.331656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5fccd-de05-44d5-bc92-3e676d5174be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:07.332399 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5fccd-de05-44d5-bc92-3e676d5174be None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:07.332780 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5fccd-de05-44d5-bc92-3e676d5174be None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:07.333666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5fccd-de05-44d5-bc92-3e676d5174be None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:07.334542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5fccd-de05-44d5-bc92-3e676d5174be None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:07.335909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5fccd-de05-44d5-bc92-3e676d5174be None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:07.345164 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01ef0d-cfa8-4379-a19f-b911d9e24d1e None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:07.377512 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c7534-29da-476a-be7b-f82ec8b06470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:07.378478 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c7534-29da-476a-be7b-f82ec8b06470 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:07.378756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c7534-29da-476a-be7b-f82ec8b06470 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:07.379032 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c7534-29da-476a-be7b-f82ec8b06470 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:07.390755 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305c7534-29da-476a-be7b-f82ec8b06470 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:28:07.399569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-305c7534-29da-476a-be7b-f82ec8b06470 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:28:07.399569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:07.399569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:07.399569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:07.399569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:07.399569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:07.399569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:07.399569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:07.399569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:07.399569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:07.399569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:07.399569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:07.399569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:28:07.399569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:28:07.399569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:28:07.399569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:28:07.400855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:28:07.400855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:28:07.400855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:28:07.400855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:28:07.400855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:07.400855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:07.400855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:07.400855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:07.400855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:07.400855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:07.400855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:07.400855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:07.400855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:07.400855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:07.400855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:07.400855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:07.401758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:07.401758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:07.401758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:07.401758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:07.401758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:28:07.401758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:28:07.401758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:28:07.401758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:28:07.401758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:28:07.401758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:28:07.401758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:28:07.401758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:07.404643 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 441/893] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 23 22:28:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:07.406702 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c7534-29da-476a-be7b-f82ec8b06470 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:07.446115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 453/894] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 22:28:07 2022] POST /identity/v3/services => generated 241 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:28:07.447529 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a783e9-4ab5-427b-acdf-110050cfedb9 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:07.447833 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a783e9-4ab5-427b-acdf-110050cfedb9 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:07.448150 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a783e9-4ab5-427b-acdf-110050cfedb9 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:07.449646 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 454/895] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:07.455268 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a1de7-1473-4e47-a753-75bf91346081 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:07.455562 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a1de7-1473-4e47-a753-75bf91346081 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:07.455854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a1de7-1473-4e47-a753-75bf91346081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:07.475206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-023a1de7-1473-4e47-a753-75bf91346081 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:07.476712 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-023a1de7-1473-4e47-a753-75bf91346081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:07.487282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:07.544484 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c7534-29da-476a-be7b-f82ec8b06470 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:07.566756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:07.569340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 455/896] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:28:07.589069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ecfaf-5ebf-40a3-96ce-5939afe0a41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:07.589277 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a1de7-1473-4e47-a753-75bf91346081 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:07.590226 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ecfaf-5ebf-40a3-96ce-5939afe0a41a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:07.590699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ecfaf-5ebf-40a3-96ce-5939afe0a41a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:07.591509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ecfaf-5ebf-40a3-96ce-5939afe0a41a None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:07.593654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ecfaf-5ebf-40a3-96ce-5939afe0a41a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:07.594298 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ecfaf-5ebf-40a3-96ce-5939afe0a41a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:07.596115 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ecfaf-5ebf-40a3-96ce-5939afe0a41a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:07.609122 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 442/897] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 23 22:28:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:07.615428 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ecfaf-5ebf-40a3-96ce-5939afe0a41a None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:07.710990 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a1de7-1473-4e47-a753-75bf91346081 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:07.732345 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ecfaf-5ebf-40a3-96ce-5939afe0a41a None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:07.757996 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc7c3f-f20c-4c9b-aa97-3d87ad9a7f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:07.759279 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc7c3f-f20c-4c9b-aa97-3d87ad9a7f41 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:07.759703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc7c3f-f20c-4c9b-aa97-3d87ad9a7f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:07.760115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc7c3f-f20c-4c9b-aa97-3d87ad9a7f41 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:07.761049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc7c3f-f20c-4c9b-aa97-3d87ad9a7f41 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:07.763106 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc7c3f-f20c-4c9b-aa97-3d87ad9a7f41 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:07.770092 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27ee27-8845-47f0-88da-7be669c01a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:07.771446 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27ee27-8845-47f0-88da-7be669c01a31 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:07.771731 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27ee27-8845-47f0-88da-7be669c01a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:07.772007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27ee27-8845-47f0-88da-7be669c01a31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:07.778168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-00fc7c3f-f20c-4c9b-aa97-3d87ad9a7f41 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 22:28:07.778168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:07.778168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:28:07.778168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:28:07.778168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:28:07.778168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:28:07.778168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:28:07.778168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:07.778168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 22:28:07.778168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:07.778168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:28:07.778168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:07.778168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:07.778168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:07.778168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:07.778168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:07.779124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:07.779124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:07.779124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:07.779124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:07.779124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:07.779124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:07.779124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:07.779124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:28:07.779124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:28:07.779124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:28:07.779124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:28:07.779124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:07.779124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:07.779124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:07.779124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:07.779124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:07.779954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:07.779954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:07.779954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:07.779954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:07.779954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:07.779954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:07.779954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:07.779954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:07.779954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:07.779954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:07.779954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:07.779954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:28:07.779954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:07.779954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 22:28:07.779954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:07.780763 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 456/898] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Jun 23 22:28:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:07.786709 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc7c3f-f20c-4c9b-aa97-3d87ad9a7f41 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:07.789607 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:07.790018 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27ee27-8845-47f0-88da-7be669c01a31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:07.791772 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27ee27-8845-47f0-88da-7be669c01a31 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:07.813845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-3b27ee27-8845-47f0-88da-7be669c01a31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:07.813845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:07.813845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:07.813845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:07.813845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:07.813845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:07.813845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:07.813845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:07.813845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:07.813845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:07.813845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:07.813845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:07.813845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:28:07.813845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:28:07.813845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:28:07.814815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:28:07.814815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:07.814815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:07.814815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:07.814815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:07.814815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:07.814815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:07.814815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:07.814815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:07.814815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:07.814815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:07.814815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:07.814815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:07.814815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:07.814815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:07.814815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:07.814815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:07.816119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:28:07.816119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:28:07.816119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:07.816119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:07.816119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 443/899] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:07.822447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27ee27-8845-47f0-88da-7be669c01a31 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:07.854718 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:07.914554 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc7c3f-f20c-4c9b-aa97-3d87ad9a7f41 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:07.947806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27ee27-8845-47f0-88da-7be669c01a31 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:07.951602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02c6fe-93db-4fe9-b625-26abc1773b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:07.952510 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02c6fe-93db-4fe9-b625-26abc1773b98 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:07.953045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02c6fe-93db-4fe9-b625-26abc1773b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:07.953529 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02c6fe-93db-4fe9-b625-26abc1773b98 None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:07.954464 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02c6fe-93db-4fe9-b625-26abc1773b98 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:07.954980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02c6fe-93db-4fe9-b625-26abc1773b98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:07.956571 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02c6fe-93db-4fe9-b625-26abc1773b98 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:07.966725 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 457/900] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 22:28:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:07.968289 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02c6fe-93db-4fe9-b625-26abc1773b98 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:07.987631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab17f1e-dcfb-4e07-be83-b5ae0a951310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:07.988840 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab17f1e-dcfb-4e07-be83-b5ae0a951310 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:07.989938 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab17f1e-dcfb-4e07-be83-b5ae0a951310 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:07.990523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab17f1e-dcfb-4e07-be83-b5ae0a951310 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:07.991606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab17f1e-dcfb-4e07-be83-b5ae0a951310 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:07.992139 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab17f1e-dcfb-4e07-be83-b5ae0a951310 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:07.993875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab17f1e-dcfb-4e07-be83-b5ae0a951310 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:08.006988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 444/901] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:08.008783 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b1b5a-6ec0-4d71-b767-3d7ba496019d None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:08.009192 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b1b5a-6ec0-4d71-b767-3d7ba496019d None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:08.009548 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b1b5a-6ec0-4d71-b767-3d7ba496019d None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:08.011010 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 445/902] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:08.012182 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b1b5a-6ec0-4d71-b767-3d7ba496019d None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:08.050394 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:08.086550 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02c6fe-93db-4fe9-b625-26abc1773b98 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:08.109555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:08.122440 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b1b5a-6ec0-4d71-b767-3d7ba496019d None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:08.123370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ab441-0004-436e-b6c9-0eb6ea3a3c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:08.124040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ab441-0004-436e-b6c9-0eb6ea3a3c99 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:08.124327 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ab441-0004-436e-b6c9-0eb6ea3a3c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:08.124635 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ab441-0004-436e-b6c9-0eb6ea3a3c99 None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:08.125357 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ab441-0004-436e-b6c9-0eb6ea3a3c99 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:08.125724 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ab441-0004-436e-b6c9-0eb6ea3a3c99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:08.127148 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ab441-0004-436e-b6c9-0eb6ea3a3c99 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:08.132676 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 458/903] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 22:28:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:08.133738 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8298b-3423-46fd-a403-3c1442d55ac4 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:08.134041 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8298b-3423-46fd-a403-3c1442d55ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:08.134320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8298b-3423-46fd-a403-3c1442d55ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:08.152013 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-f1c8298b-3423-46fd-a403-3c1442d55ac4 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:08.153275 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-f1c8298b-3423-46fd-a403-3c1442d55ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:08.164101 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d30916e-27ad-4a55-881b-5369791fc737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:08.165165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30916e-27ad-4a55-881b-5369791fc737 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:08.165344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30916e-27ad-4a55-881b-5369791fc737 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:08.165871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30916e-27ad-4a55-881b-5369791fc737 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:08.173862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d30916e-27ad-4a55-881b-5369791fc737 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:08.175438 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d30916e-27ad-4a55-881b-5369791fc737 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:08.192774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-6d30916e-27ad-4a55-881b-5369791fc737 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:28:08.192774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:08.192774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:08.192774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:08.192774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:08.192774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:08.192774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:08.192774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:08.192774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:08.192774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:08.192774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:08.192774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:08.192774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:28:08.192774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:28:08.192774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:28:08.192774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:28:08.194035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:08.194035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:08.194035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:28:08.194035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:28:08.194035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:28:08.194035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:28:08.194035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:08.194035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:08.194035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:08.194035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:08.194035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:08.194035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:08.194035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:08.194035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:08.194035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:08.194035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:08.194035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:08.194035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:08.195002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:08.195002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:08.195002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:28:08.195002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:28:08.195002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:28:08.195002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:28:08.195002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:28:08.195002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:28:08.195002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:28:08.195002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:08.197303 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 446/904] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 23 22:28:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:08.198309 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d30916e-27ad-4a55-881b-5369791fc737 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:08.231747 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 459/905] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:08.232910 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8298b-3423-46fd-a403-3c1442d55ac4 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:08.308495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:08.313266 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d30916e-27ad-4a55-881b-5369791fc737 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:08.352285 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8298b-3423-46fd-a403-3c1442d55ac4 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:08.354041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025bad87-87cc-4443-ad87-6942eca35a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:08.355801 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bad87-87cc-4443-ad87-6942eca35a29 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:08.356281 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bad87-87cc-4443-ad87-6942eca35a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:08.356787 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bad87-87cc-4443-ad87-6942eca35a29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:08.357765 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025bad87-87cc-4443-ad87-6942eca35a29 None admin] RBAC: Adding query filter params (interface=public, service_id=bbea1fe4c5f14dd9b29a2b65362b302c, region_id=RegionOne) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:08.358267 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025bad87-87cc-4443-ad87-6942eca35a29 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:08.360479 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025bad87-87cc-4443-ad87-6942eca35a29 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:08.370830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 447/906] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 23 22:28:08 2022] GET /identity/v3/endpoints?service_id=bbea1fe4c5f14dd9b29a2b65362b302c&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:08.371773 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025bad87-87cc-4443-ad87-6942eca35a29 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:08.387910 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:08.393098 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b20344-fce5-4642-9152-9e5e61f94948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:08.393878 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b20344-fce5-4642-9152-9e5e61f94948 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:08.394163 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b20344-fce5-4642-9152-9e5e61f94948 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:08.394439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b20344-fce5-4642-9152-9e5e61f94948 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:08.395395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b20344-fce5-4642-9152-9e5e61f94948 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:08.396312 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b20344-fce5-4642-9152-9e5e61f94948 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:08.397898 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b20344-fce5-4642-9152-9e5e61f94948 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:08.418225 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 460/907] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 23 22:28:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:08.419291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b20344-fce5-4642-9152-9e5e61f94948 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:08.479332 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025bad87-87cc-4443-ad87-6942eca35a29 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:08.521891 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400c3e7-b5af-4766-beb2-aed9526832b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:08.523224 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400c3e7-b5af-4766-beb2-aed9526832b3 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:08.523699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400c3e7-b5af-4766-beb2-aed9526832b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:08.524044 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400c3e7-b5af-4766-beb2-aed9526832b3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:08.524796 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7400c3e7-b5af-4766-beb2-aed9526832b3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:08.525998 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b20344-fce5-4642-9152-9e5e61f94948 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:08.526211 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7400c3e7-b5af-4766-beb2-aed9526832b3 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:08.539155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-7400c3e7-b5af-4766-beb2-aed9526832b3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 22:28:08.539155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:08.539155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:28:08.539155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:28:08.539155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:28:08.539155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:28:08.539155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:28:08.539155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:08.539155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 22:28:08.539155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:08.539155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:28:08.539155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:08.539155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:08.539155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:08.539155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:08.539155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:08.539155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:08.540169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:08.540169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:08.540169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:08.540169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:08.540169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:08.540169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:08.540169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:28:08.540169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:28:08.540169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:28:08.540169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:28:08.540169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:08.540169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:08.540169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:08.540169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:08.540169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:08.540169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:08.541637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:08.541637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:08.541637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:08.541637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:08.541637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:08.541637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:08.541637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:08.541637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:08.541637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:08.541637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:08.541637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:28:08.541637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:08.541637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 22:28:08.541637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:08.541637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 448/908] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:28:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:08.542464 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400c3e7-b5af-4766-beb2-aed9526832b3 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:08.550472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:08.574025 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad2ad3-57f3-4033-850e-23a8f2a7727b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:08.575339 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad2ad3-57f3-4033-850e-23a8f2a7727b None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:08.576090 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad2ad3-57f3-4033-850e-23a8f2a7727b None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:08.576679 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad2ad3-57f3-4033-850e-23a8f2a7727b None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:08.577608 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad2ad3-57f3-4033-850e-23a8f2a7727b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:08.580128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad2ad3-57f3-4033-850e-23a8f2a7727b None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:08.589010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 461/909] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 23 22:28:08 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:28:08.590141 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad2ad3-57f3-4033-850e-23a8f2a7727b None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:08.626769 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:08.655656 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400c3e7-b5af-4766-beb2-aed9526832b3 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:08.697765 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a232f40-571d-4ac3-96e3-07ba34b6c9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:08.698688 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a232f40-571d-4ac3-96e3-07ba34b6c9ca None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:08.699108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a232f40-571d-4ac3-96e3-07ba34b6c9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:08.699516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a232f40-571d-4ac3-96e3-07ba34b6c9ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:08.725542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad2ad3-57f3-4033-850e-23a8f2a7727b None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:08.764379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0cd35-5e58-464c-8022-0a88cd125ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:08.765084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0cd35-5e58-464c-8022-0a88cd125ccd None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:08.765370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0cd35-5e58-464c-8022-0a88cd125ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:08.765645 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0cd35-5e58-464c-8022-0a88cd125ccd None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:08.766351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0cd35-5e58-464c-8022-0a88cd125ccd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:08.766656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0cd35-5e58-464c-8022-0a88cd125ccd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:08.768013 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0cd35-5e58-464c-8022-0a88cd125ccd None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:08.773749 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 462/910] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:28:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:08.778978 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a232f40-571d-4ac3-96e3-07ba34b6c9ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:08.780497 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0cd35-5e58-464c-8022-0a88cd125ccd None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:08.781622 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a232f40-571d-4ac3-96e3-07ba34b6c9ca None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:08.837399 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:08.881744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:08.895327 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0cd35-5e58-464c-8022-0a88cd125ccd None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:08.916659 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 449/911] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:28:08 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:28:08.919959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a232f40-571d-4ac3-96e3-07ba34b6c9ca None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:08.940062 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee14d41-a86d-4e2f-bad8-40913ec8aac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:08.940764 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee14d41-a86d-4e2f-bad8-40913ec8aac3 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:08.941051 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee14d41-a86d-4e2f-bad8-40913ec8aac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:08.941325 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee14d41-a86d-4e2f-bad8-40913ec8aac3 None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:08.942037 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee14d41-a86d-4e2f-bad8-40913ec8aac3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:08.942945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee14d41-a86d-4e2f-bad8-40913ec8aac3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:08.945055 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee14d41-a86d-4e2f-bad8-40913ec8aac3 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:08.952611 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 463/912] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 23 22:28:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:08.957637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee14d41-a86d-4e2f-bad8-40913ec8aac3 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:09.044049 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a232f40-571d-4ac3-96e3-07ba34b6c9ca None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:09.075836 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee14d41-a86d-4e2f-bad8-40913ec8aac3 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:09.088841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:09.092299 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7f325-74f8-4fba-8b9b-a902bfadb15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:09.093712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7f325-74f8-4fba-8b9b-a902bfadb15b None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:09.094548 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7f325-74f8-4fba-8b9b-a902bfadb15b None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:09.095006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7f325-74f8-4fba-8b9b-a902bfadb15b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:09.109543 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7f325-74f8-4fba-8b9b-a902bfadb15b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:09.111916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7f325-74f8-4fba-8b9b-a902bfadb15b None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:09.115698 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7789a40-6ee4-46cb-a165-1949bf00b178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:09.116536 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7789a40-6ee4-46cb-a165-1949bf00b178 None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:09.116915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7789a40-6ee4-46cb-a165-1949bf00b178 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:09.117264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7789a40-6ee4-46cb-a165-1949bf00b178 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:09.118137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7789a40-6ee4-46cb-a165-1949bf00b178 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:09.119729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7789a40-6ee4-46cb-a165-1949bf00b178 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:09.140726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-bec7f325-74f8-4fba-8b9b-a902bfadb15b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:09.140726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:09.140726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:09.140726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:09.140726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:09.140726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:09.140726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:09.140726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:09.140726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:09.140726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:09.140726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:09.140726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:09.140726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:28:09.140726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:28:09.140726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:28:09.141735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:28:09.141735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:09.141735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:09.141735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:09.141735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:09.141735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:09.141735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:09.141735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:09.141735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:09.141735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:09.141735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:09.141735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:09.141735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:09.141735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:09.141735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:09.141735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:09.141735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:09.142657 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:28:09.142657 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:28:09.142657 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:09.142657 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:09.142657 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 450/913] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:09.143850 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7f325-74f8-4fba-8b9b-a902bfadb15b None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:09.174362 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:09.205225 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 464/914] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 23 22:28:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:28:09.261402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7f325-74f8-4fba-8b9b-a902bfadb15b None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:09.309035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d0294-36b8-4a62-99b1-e5f8c5501b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:09.309713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d0294-36b8-4a62-99b1-e5f8c5501b69 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:09.309997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d0294-36b8-4a62-99b1-e5f8c5501b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:09.310277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d0294-36b8-4a62-99b1-e5f8c5501b69 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:09.311014 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d0294-36b8-4a62-99b1-e5f8c5501b69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:09.311323 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d0294-36b8-4a62-99b1-e5f8c5501b69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:09.312806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d0294-36b8-4a62-99b1-e5f8c5501b69 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:09.326391 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 451/915] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:09.334366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7789a40-6ee4-46cb-a165-1949bf00b178 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:09.446532 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7789a40-6ee4-46cb-a165-1949bf00b178 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:09.483248 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ae393-2562-4bce-a159-1abdb5aacf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:09.484668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ae393-2562-4bce-a159-1abdb5aacf71 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:09.485161 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ae393-2562-4bce-a159-1abdb5aacf71 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:09.485639 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ae393-2562-4bce-a159-1abdb5aacf71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:09.493136 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ae393-2562-4bce-a159-1abdb5aacf71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:09.494797 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ae393-2562-4bce-a159-1abdb5aacf71 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:09.509132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-369ae393-2562-4bce-a159-1abdb5aacf71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:28:09.509132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:09.509132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:09.509132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:09.509132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:09.509132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:09.509132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:09.509132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:09.509132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:09.509132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:09.509132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:09.509132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:09.509132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:28:09.509132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:28:09.509132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:28:09.510045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:28:09.510045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:09.510045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:09.510045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:09.510045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:09.510045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:09.510045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:09.510045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:09.510045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:09.510045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:09.510045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:09.510045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:09.510045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:09.510045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:09.510045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:09.510045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:09.510045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:09.510934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:28:09.510934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:28:09.510934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:28:09.510934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:28:09.510934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:28:09.510934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:28:09.510934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:28:09.510934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:09.513383 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 465/916] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:28:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:09.519328 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d0294-36b8-4a62-99b1-e5f8c5501b69 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:09.612566 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:09.619782 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ecb58-8219-4aaf-931f-3951d75f7a8b None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:09.620072 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ecb58-8219-4aaf-931f-3951d75f7a8b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:09.620381 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ecb58-8219-4aaf-931f-3951d75f7a8b None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:09.637122 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 466/917] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:28:09.637122 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a71765-9fc0-4b45-a353-820db16d065e None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:09.637122 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a71765-9fc0-4b45-a353-820db16d065e None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:09.637122 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a71765-9fc0-4b45-a353-820db16d065e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:09.637550 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d0294-36b8-4a62-99b1-e5f8c5501b69 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:09.652103 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-93a71765-9fc0-4b45-a353-820db16d065e None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:09.654000 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-93a71765-9fc0-4b45-a353-820db16d065e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:09.671756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d24f47-c5ea-4a08-a9bb-abcda4b1458d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:09.672420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d24f47-c5ea-4a08-a9bb-abcda4b1458d None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:09.672751 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d24f47-c5ea-4a08-a9bb-abcda4b1458d None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:09.673032 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d24f47-c5ea-4a08-a9bb-abcda4b1458d None admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:09.673827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d24f47-c5ea-4a08-a9bb-abcda4b1458d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:09.674144 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d24f47-c5ea-4a08-a9bb-abcda4b1458d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:09.675590 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d24f47-c5ea-4a08-a9bb-abcda4b1458d None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:09.681353 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:09.688690 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 452/918] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:28:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:09.695345 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d24f47-c5ea-4a08-a9bb-abcda4b1458d None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:09.748663 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 467/919] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:09.766254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a71765-9fc0-4b45-a353-820db16d065e None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:09.806442 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d24f47-c5ea-4a08-a9bb-abcda4b1458d None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:09.841510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bd49d-6c13-4329-ae32-58f6085a54c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:09.842169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bd49d-6c13-4329-ae32-58f6085a54c0 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:09.842453 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bd49d-6c13-4329-ae32-58f6085a54c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:09.842730 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bd49d-6c13-4329-ae32-58f6085a54c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:09.843562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bd49d-6c13-4329-ae32-58f6085a54c0 None admin] RBAC: Adding query filter params (role.id=b1cc8597c7b144ed9b6bed9fc40f3c97, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=0a7c204a298c42f0bbc32fa9a6ce1f38) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:09.843904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bd49d-6c13-4329-ae32-58f6085a54c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:09.845345 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bd49d-6c13-4329-ae32-58f6085a54c0 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:09.851502 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 453/920] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:28:09 2022] GET /identity/v3/role_assignments?user.id=0a7c204a298c42f0bbc32fa9a6ce1f38&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=b1cc8597c7b144ed9b6bed9fc40f3c97 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:09.879298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a71765-9fc0-4b45-a353-820db16d065e None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:09.920766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f20926-90ca-4ba7-b3e8-e5736670e185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:09.921595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f20926-90ca-4ba7-b3e8-e5736670e185 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:09.921880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f20926-90ca-4ba7-b3e8-e5736670e185 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:09.922156 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f20926-90ca-4ba7-b3e8-e5736670e185 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:09.922876 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f20926-90ca-4ba7-b3e8-e5736670e185 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:09.924284 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f20926-90ca-4ba7-b3e8-e5736670e185 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:09.937656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-e9f20926-90ca-4ba7-b3e8-e5736670e185 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:28:09.937656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:09.937656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:28:09.937656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:28:09.937656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:28:09.937656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:28:09.937656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:28:09.937656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:09.937656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:28:09.937656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:09.937656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:28:09.937656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:09.937656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:09.937656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:09.937656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:09.937656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:09.937656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:09.938701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:09.938701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:09.938701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:09.938701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:09.938701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:09.938701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:09.938701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:28:09.938701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:28:09.938701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:28:09.938701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:28:09.938701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:09.938701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:09.938701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:09.938701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:09.938701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:09.938701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:09.939544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:09.939544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:09.939544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:09.939544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:09.939544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:09.939544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:09.939544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:09.939544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:09.939544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:09.939544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:09.939544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:28:09.939544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:09.939544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:28:09.939544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:09.939544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 468/921] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 22:28:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:09.945946 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bd49d-6c13-4329-ae32-58f6085a54c0 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:09.960967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:10.056550 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bd49d-6c13-4329-ae32-58f6085a54c0 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:10.090933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab48c9-b61f-4396-871e-01cf2d878949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:10.091921 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab48c9-b61f-4396-871e-01cf2d878949 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:10.092410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab48c9-b61f-4396-871e-01cf2d878949 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:10.092939 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab48c9-b61f-4396-871e-01cf2d878949 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:10.093843 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab48c9-b61f-4396-871e-01cf2d878949 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:10.094355 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab48c9-b61f-4396-871e-01cf2d878949 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:10.095921 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab48c9-b61f-4396-871e-01cf2d878949 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:10.102144 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 454/922] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 23 22:28:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:10.108031 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f20926-90ca-4ba7-b3e8-e5736670e185 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:10.132120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:10.215750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f20926-90ca-4ba7-b3e8-e5736670e185 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:10.250608 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120444d-0c4e-4436-a98f-ccb248db0db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:10.251642 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120444d-0c4e-4436-a98f-ccb248db0db4 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:10.252129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120444d-0c4e-4436-a98f-ccb248db0db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:10.252626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120444d-0c4e-4436-a98f-ccb248db0db4 None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:10.253599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120444d-0c4e-4436-a98f-ccb248db0db4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:10.254102 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120444d-0c4e-4436-a98f-ccb248db0db4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:10.255689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120444d-0c4e-4436-a98f-ccb248db0db4 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:10.261518 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 469/923] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 23 22:28:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:10.267467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab48c9-b61f-4396-871e-01cf2d878949 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:10.350898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:10.378784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab48c9-b61f-4396-871e-01cf2d878949 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:10.415205 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b90b4-c3a5-41fb-9e02-1a60c94713a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:10.416502 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b90b4-c3a5-41fb-9e02-1a60c94713a2 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:10.416963 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b90b4-c3a5-41fb-9e02-1a60c94713a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:10.417390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b90b4-c3a5-41fb-9e02-1a60c94713a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:10.418362 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b90b4-c3a5-41fb-9e02-1a60c94713a2 None admin] RBAC: Adding query filter params (interface=public, service_id=39274e290c77462ba56e8f8e776fd4e9, region_id=RegionOne) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:10.418828 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b90b4-c3a5-41fb-9e02-1a60c94713a2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:10.420414 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b90b4-c3a5-41fb-9e02-1a60c94713a2 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:10.427087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 455/924] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 23 22:28:10 2022] GET /identity/v3/endpoints?service_id=39274e290c77462ba56e8f8e776fd4e9&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:10.436602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120444d-0c4e-4436-a98f-ccb248db0db4 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:10.536990 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:10.539969 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120444d-0c4e-4436-a98f-ccb248db0db4 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:10.575200 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd89d2-fe8e-4653-9bbc-4ff9a184e28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:10.575875 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd89d2-fe8e-4653-9bbc-4ff9a184e28f None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:10.576216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd89d2-fe8e-4653-9bbc-4ff9a184e28f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:10.576504 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd89d2-fe8e-4653-9bbc-4ff9a184e28f None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:10.577282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd89d2-fe8e-4653-9bbc-4ff9a184e28f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:10.579068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd89d2-fe8e-4653-9bbc-4ff9a184e28f None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:10.585511 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 470/925] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 23 22:28:10 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:28:10.778577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940e2b1-3403-465e-a56e-06ac23d2e439 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:10.779269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940e2b1-3403-465e-a56e-06ac23d2e439 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:10.779754 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940e2b1-3403-465e-a56e-06ac23d2e439 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:10.781652 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 456/926] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:10.787675 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b31fac-d891-46e9-bc8a-cfdc8a90ca72 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:10.787964 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b31fac-d891-46e9-bc8a-cfdc8a90ca72 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:10.788240 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b31fac-d891-46e9-bc8a-cfdc8a90ca72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:10.806189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-a0b31fac-d891-46e9-bc8a-cfdc8a90ca72 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:10.807433 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a0b31fac-d891-46e9-bc8a-cfdc8a90ca72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:10.877424 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 471/927] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:10.901003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940e2b1-3403-465e-a56e-06ac23d2e439 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:11.012416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940e2b1-3403-465e-a56e-06ac23d2e439 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:11.050160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4663b2-3d1a-482b-98cd-ebf2a636e08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:11.051326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4663b2-3d1a-482b-98cd-ebf2a636e08b None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:11.051873 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4663b2-3d1a-482b-98cd-ebf2a636e08b None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:11.052348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4663b2-3d1a-482b-98cd-ebf2a636e08b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:11.053247 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4663b2-3d1a-482b-98cd-ebf2a636e08b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:11.054844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4663b2-3d1a-482b-98cd-ebf2a636e08b None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:11.067764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-4b4663b2-3d1a-482b-98cd-ebf2a636e08b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 22:28:11.067764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:11.067764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:28:11.067764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:28:11.067764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:28:11.067764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:28:11.067764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:28:11.067764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:11.067764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 22:28:11.067764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:11.067764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:28:11.067764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:11.067764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:11.067764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:11.067764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:11.067764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:11.068761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:11.068761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:11.068761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:11.068761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:11.068761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:11.068761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:11.068761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:11.068761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:28:11.068761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:28:11.068761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:28:11.068761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:28:11.068761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:11.068761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:11.068761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:11.068761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:11.068761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:11.069621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:11.069621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:11.069621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:11.069621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:11.069621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:11.069621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:11.069621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:11.069621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:11.069621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:11.069621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:11.069621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:11.069621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:28:11.069621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:11.069621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 22:28:11.069621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:11.076650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:11.076928 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 457/928] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Jun 23 22:28:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:11.078981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b31fac-d891-46e9-bc8a-cfdc8a90ca72 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:11.160849 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:11.183341 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b31fac-d891-46e9-bc8a-cfdc8a90ca72 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:11.218220 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308e462-44da-4db0-bf93-148db89a46aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:11.218897 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308e462-44da-4db0-bf93-148db89a46aa None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:11.219235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308e462-44da-4db0-bf93-148db89a46aa None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:11.219523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308e462-44da-4db0-bf93-148db89a46aa None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:11.220236 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0308e462-44da-4db0-bf93-148db89a46aa None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:11.220548 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0308e462-44da-4db0-bf93-148db89a46aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:11.221940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0308e462-44da-4db0-bf93-148db89a46aa None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:11.227762 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 472/929] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 22:28:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:11.233575 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4663b2-3d1a-482b-98cd-ebf2a636e08b None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:11.347799 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4663b2-3d1a-482b-98cd-ebf2a636e08b None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:11.377150 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a06243-595b-44d5-81a4-44e323037079 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:11.377464 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a06243-595b-44d5-81a4-44e323037079 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:11.377712 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a06243-595b-44d5-81a4-44e323037079 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:11.379276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 473/930] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:11.385082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099f2ef-5cee-41dd-bc04-d8f16eeca5e0 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:11.385372 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099f2ef-5cee-41dd-bc04-d8f16eeca5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:11.385660 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099f2ef-5cee-41dd-bc04-d8f16eeca5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:11.386404 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8106268-42ad-4cff-b2a9-b87e68093f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:11.387307 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8106268-42ad-4cff-b2a9-b87e68093f94 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:11.387737 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8106268-42ad-4cff-b2a9-b87e68093f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:11.388170 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8106268-42ad-4cff-b2a9-b87e68093f94 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:11.389106 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8106268-42ad-4cff-b2a9-b87e68093f94 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:11.389418 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8106268-42ad-4cff-b2a9-b87e68093f94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:11.390843 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8106268-42ad-4cff-b2a9-b87e68093f94 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:11.399243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 458/931] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 22:28:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:11.406341 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8106268-42ad-4cff-b2a9-b87e68093f94 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:11.409455 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-6099f2ef-5cee-41dd-bc04-d8f16eeca5e0 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:11.411090 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6099f2ef-5cee-41dd-bc04-d8f16eeca5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:11.480933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:11.488598 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 474/932] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:28:11.507317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099f2ef-5cee-41dd-bc04-d8f16eeca5e0 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:11.530041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8106268-42ad-4cff-b2a9-b87e68093f94 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:11.568271 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821e0e8-89da-4038-bd5e-fbe374e315a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:11.569297 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821e0e8-89da-4038-bd5e-fbe374e315a5 None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:11.569769 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821e0e8-89da-4038-bd5e-fbe374e315a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:11.570253 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821e0e8-89da-4038-bd5e-fbe374e315a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:11.571176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821e0e8-89da-4038-bd5e-fbe374e315a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:11.572939 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821e0e8-89da-4038-bd5e-fbe374e315a5 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:11.580538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:11.610551 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099f2ef-5cee-41dd-bc04-d8f16eeca5e0 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:11.649781 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44b508-cc81-4169-9476-1102c83bfdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:11.650904 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44b508-cc81-4169-9476-1102c83bfdcf None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:11.651479 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44b508-cc81-4169-9476-1102c83bfdcf None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:11.651957 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44b508-cc81-4169-9476-1102c83bfdcf None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:11.653173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44b508-cc81-4169-9476-1102c83bfdcf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:11.656097 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44b508-cc81-4169-9476-1102c83bfdcf None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:11.656759 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 459/933] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 23 22:28:11 2022] POST /identity/v3/endpoints => generated 347 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:28:11.657309 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: WARNING keystone.server.flask.common [None req-0f44b508-cc81-4169-9476-1102c83bfdcf None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 23 22:28:11.701781 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.sql.core [None req-0f44b508-cc81-4169-9476-1102c83bfdcf None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 22:28:11.701781 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 22:28:11.701781 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [parameters: {'id': 'c6229d13bf434cfda0a719a200575d9e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 22:28:11.701781 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80518) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 22:28:11.721354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-0f44b508-cc81-4169-9476-1102c83bfdcf None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c6229d13bf434cfda0a719a200575d9e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c6229d13bf434cfda0a719a200575d9e. Jun 23 22:28:11.721354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:11.721354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 22:28:11.721354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 22:28:11.721354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 22:28:11.721354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 22:28:11.721354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 22:28:11.721354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 22:28:11.721354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 22:28:11.721354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application conn.query(q) Jun 23 22:28:11.721354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 22:28:11.721354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 22:28:11.721354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 22:28:11.722252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application result.read() Jun 23 22:28:11.722252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 22:28:11.722252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 22:28:11.722252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 22:28:11.722252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 22:28:11.722252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 22:28:11.722252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 22:28:11.722252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 22:28:11.722252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 22:28:11.722252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 22:28:11.722252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:11.722252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 22:28:11.722252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:11.722252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:11.722252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 22:28:11.722252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 22:28:11.722252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 23 22:28:11.722252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 23 22:28:11.724278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 22:28:11.724278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application next(self.gen) Jun 23 22:28:11.724278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 22:28:11.724278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application yield resource Jun 23 22:28:11.724278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 22:28:11.724278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application next(self.gen) Jun 23 22:28:11.724278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 22:28:11.724278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 22:28:11.724278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 22:28:11.724278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 22:28:11.724278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 22:28:11.724278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise self.value Jun 23 22:28:11.724278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 22:28:11.724278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 22:28:11.724278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 22:28:11.724278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application session.commit() Jun 23 22:28:11.724278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 22:28:11.725217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 22:28:11.725217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 22:28:11.725217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 22:28:11.725217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 22:28:11.725217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self.session.flush() Jun 23 22:28:11.725217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 22:28:11.725217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 22:28:11.725217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 22:28:11.725217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 22:28:11.725217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 22:28:11.725217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application compat.raise_( Jun 23 22:28:11.725217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 22:28:11.725217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception Jun 23 22:28:11.725217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 22:28:11.725217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 22:28:11.725217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 22:28:11.725217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 22:28:11.726104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 22:28:11.726104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 22:28:11.726104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 22:28:11.726104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 22:28:11.726104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 22:28:11.726104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 22:28:11.726104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 22:28:11.726104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 22:28:11.726104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 22:28:11.726104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 22:28:11.726104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 22:28:11.726104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 22:28:11.726104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 22:28:11.726104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 22:28:11.726104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 22:28:11.726880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 22:28:11.726880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 22:28:11.726880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception Jun 23 22:28:11.726880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 22:28:11.726880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 22:28:11.726880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 22:28:11.726880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 22:28:11.726880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 22:28:11.726880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 22:28:11.726880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 22:28:11.726880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application conn.query(q) Jun 23 22:28:11.726880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 22:28:11.726880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 22:28:11.726880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 22:28:11.726880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application result.read() Jun 23 22:28:11.726880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 22:28:11.726880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 22:28:11.727844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 22:28:11.727844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 22:28:11.727844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 22:28:11.727844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 22:28:11.727844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 22:28:11.727844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 22:28:11.727844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 22:28:11.727844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 22:28:11.727844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [parameters: {'id': 'c6229d13bf434cfda0a719a200575d9e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 22:28:11.727844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 22:28:11.727844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:11.727844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:28:11.727844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:11.727844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:11.727844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 23 22:28:11.729894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 23 22:28:11.729894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 22:28:11.729894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 22:28:11.729894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 23 22:28:11.729894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:11.729894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:28:11.729894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:11.729894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:11.729894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:11.729894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:11.729894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:11.729894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:11.729894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:11.729894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:11.729894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:11.729894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:11.729894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:11.732288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:11.732288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 23 22:28:11.732288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 23 22:28:11.732288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:11.732288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:11.732288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 23 22:28:11.732288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 23 22:28:11.732288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c6229d13bf434cfda0a719a200575d9e. Jun 23 22:28:11.732288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:11.735732 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 475/934] 10.222.0.33 () {60 vars in 1140 bytes} [Thu Jun 23 22:28:11 2022] POST /identity/v3/projects => generated 262 bytes in 229 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:11.742153 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821e0e8-89da-4038-bd5e-fbe374e315a5 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:11.861536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821e0e8-89da-4038-bd5e-fbe374e315a5 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:11.878785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:11.898634 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dcdd6f-1ef4-4d8c-a052-ae9f63bc3935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:11.899501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dcdd6f-1ef4-4d8c-a052-ae9f63bc3935 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:11.899788 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dcdd6f-1ef4-4d8c-a052-ae9f63bc3935 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:11.900180 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dcdd6f-1ef4-4d8c-a052-ae9f63bc3935 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:11.910926 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dcdd6f-1ef4-4d8c-a052-ae9f63bc3935 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:11.912963 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dcdd6f-1ef4-4d8c-a052-ae9f63bc3935 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:11.932878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-32dcdd6f-1ef4-4d8c-a052-ae9f63bc3935 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:28:11.932878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:11.932878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:11.932878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:11.932878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:11.932878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:11.932878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:11.932878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:11.932878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:11.932878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:11.932878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:11.932878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:11.932878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:28:11.932878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:28:11.932878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:28:11.933919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:28:11.933919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:11.933919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:11.933919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:11.933919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:11.933919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:11.933919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:11.933919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:11.933919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:11.933919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:11.933919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:11.933919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:11.933919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:11.933919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:11.933919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:11.933919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:11.933919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:11.935172 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:28:11.935172 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:28:11.935172 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:28:11.935172 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:28:11.935172 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:28:11.935172 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:28:11.935172 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:28:11.935172 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:11.939605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 460/935] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:28:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:11.942227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44b508-cc81-4169-9476-1102c83bfdcf None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:11.976693 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9df3db-60d2-4181-a913-28fac230f545 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:11.976980 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9df3db-60d2-4181-a913-28fac230f545 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:11.977264 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9df3db-60d2-4181-a913-28fac230f545 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:11.978887 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 461/936] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:11.983794 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89cae4-7d5d-4f8d-b1d0-18d50c7925b9 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:11.984091 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89cae4-7d5d-4f8d-b1d0-18d50c7925b9 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:11.984376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89cae4-7d5d-4f8d-b1d0-18d50c7925b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:12.002956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-8b89cae4-7d5d-4f8d-b1d0-18d50c7925b9 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:12.004857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-8b89cae4-7d5d-4f8d-b1d0-18d50c7925b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:12.067337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44b508-cc81-4169-9476-1102c83bfdcf None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:12.095088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:12.104672 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 462/937] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:12.110350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968d779-f26b-4d4f-9a5c-42ca253ce804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:12.111069 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968d779-f26b-4d4f-9a5c-42ca253ce804 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:12.111360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968d779-f26b-4d4f-9a5c-42ca253ce804 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:12.111693 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968d779-f26b-4d4f-9a5c-42ca253ce804 None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:12.112503 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968d779-f26b-4d4f-9a5c-42ca253ce804 None admin] RBAC: Adding query filter params (name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:12.112956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968d779-f26b-4d4f-9a5c-42ca253ce804 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:12.115139 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968d779-f26b-4d4f-9a5c-42ca253ce804 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:12.127796 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89cae4-7d5d-4f8d-b1d0-18d50c7925b9 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:12.140423 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 476/938] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 22:28:11 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:12.251878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89cae4-7d5d-4f8d-b1d0-18d50c7925b9 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:12.261010 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:12.296609 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951f64b-0346-4e6a-8b16-759e34dde360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:12.296609 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951f64b-0346-4e6a-8b16-759e34dde360 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:12.296609 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951f64b-0346-4e6a-8b16-759e34dde360 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:12.296609 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951f64b-0346-4e6a-8b16-759e34dde360 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:12.305320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951f64b-0346-4e6a-8b16-759e34dde360 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:28:12.314176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-2951f64b-0346-4e6a-8b16-759e34dde360 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:28:12.314176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:12.314176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:12.314176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:12.314176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:12.314176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:12.314176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:12.314176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:12.314176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:12.314176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:12.314176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:12.314176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:12.314176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:28:12.314176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:28:12.314176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:28:12.314176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:28:12.315330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:28:12.315330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:28:12.315330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:28:12.315330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:28:12.315330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:12.315330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:12.315330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:12.315330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:12.315330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:12.315330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:12.315330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:12.315330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:12.315330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:12.315330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:12.315330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:12.315330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:12.316385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:12.316385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:12.316385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:12.316385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:12.316385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:28:12.316385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:28:12.316385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:28:12.316385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:28:12.316385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:28:12.316385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:28:12.316385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:28:12.316385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:12.320375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 463/939] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 23 22:28:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:12.326271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968d779-f26b-4d4f-9a5c-42ca253ce804 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:12.436787 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968d779-f26b-4d4f-9a5c-42ca253ce804 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:12.473227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ec8e0-e22b-495f-a2c9-c024b27ca1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:12.474260 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ec8e0-e22b-495f-a2c9-c024b27ca1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:12.474732 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ec8e0-e22b-495f-a2c9-c024b27ca1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:12.475194 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ec8e0-e22b-495f-a2c9-c024b27ca1c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:12.476139 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055ec8e0-e22b-495f-a2c9-c024b27ca1c3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:12.476717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055ec8e0-e22b-495f-a2c9-c024b27ca1c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:12.478269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055ec8e0-e22b-495f-a2c9-c024b27ca1c3 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:12.487282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 477/940] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 23 22:28:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:12.493402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951f64b-0346-4e6a-8b16-759e34dde360 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:12.600859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951f64b-0346-4e6a-8b16-759e34dde360 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:12.636006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc1e6b-16d6-4c0c-9b0a-94329dc7e5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:12.637126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc1e6b-16d6-4c0c-9b0a-94329dc7e5fb None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:12.637589 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc1e6b-16d6-4c0c-9b0a-94329dc7e5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:12.638061 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc1e6b-16d6-4c0c-9b0a-94329dc7e5fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:12.649238 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc1e6b-16d6-4c0c-9b0a-94329dc7e5fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:12.650958 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc1e6b-16d6-4c0c-9b0a-94329dc7e5fb None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:12.668768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-d7dc1e6b-16d6-4c0c-9b0a-94329dc7e5fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:12.668768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:12.668768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:12.668768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:12.668768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:12.668768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:12.668768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:12.668768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:12.668768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:12.668768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:12.668768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:12.668768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:12.668768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:28:12.668768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:28:12.668768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:28:12.669722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:28:12.669722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:12.669722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:12.669722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:12.669722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:12.669722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:12.669722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:12.669722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:12.669722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:12.669722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:12.669722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:12.669722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:12.669722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:12.669722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:12.669722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:12.669722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:12.669722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:12.670610 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:28:12.670610 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:28:12.670610 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:12.670610 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:12.672916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 464/941] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:12.679295 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ec8e0-e22b-495f-a2c9-c024b27ca1c3 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:12.701889 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c79a8c-3c42-47ac-831a-2a932089249e None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:12.702173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c79a8c-3c42-47ac-831a-2a932089249e None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:12.702382 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c79a8c-3c42-47ac-831a-2a932089249e None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:12.703871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 465/942] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:28:12.709349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fd86e-5aa2-40e9-b096-a4746c909182 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:12.709637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fd86e-5aa2-40e9-b096-a4746c909182 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:12.709921 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fd86e-5aa2-40e9-b096-a4746c909182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:12.733351 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-7b7fd86e-5aa2-40e9-b096-a4746c909182 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:12.734691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-7b7fd86e-5aa2-40e9-b096-a4746c909182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:12.741805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:12.754154 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:12.796159 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ec8e0-e22b-495f-a2c9-c024b27ca1c3 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:12.827124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 466/943] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:12.839135 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e903a-2070-4f3d-93fe-39af3c7a873f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:12.840176 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e903a-2070-4f3d-93fe-39af3c7a873f None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:12.840727 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e903a-2070-4f3d-93fe-39af3c7a873f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:12.841374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e903a-2070-4f3d-93fe-39af3c7a873f None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:12.842321 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e903a-2070-4f3d-93fe-39af3c7a873f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:12.842833 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e903a-2070-4f3d-93fe-39af3c7a873f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:12.844503 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e903a-2070-4f3d-93fe-39af3c7a873f None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:12.852379 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fd86e-5aa2-40e9-b096-a4746c909182 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:12.863805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 478/944] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:12.870043 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e903a-2070-4f3d-93fe-39af3c7a873f None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:12.966525 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fd86e-5aa2-40e9-b096-a4746c909182 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:12.977384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e903a-2070-4f3d-93fe-39af3c7a873f None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:12.988720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:13.004928 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938ef93-d986-4efb-a72f-77e055625130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:13.006239 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938ef93-d986-4efb-a72f-77e055625130 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:13.006713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938ef93-d986-4efb-a72f-77e055625130 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:13.013027 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938ef93-d986-4efb-a72f-77e055625130 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:13.013027 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938ef93-d986-4efb-a72f-77e055625130 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:13.013027 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938ef93-d986-4efb-a72f-77e055625130 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:13.014503 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b2c19-a95d-4542-b614-7cc3f53a780b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:13.015322 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b2c19-a95d-4542-b614-7cc3f53a780b None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:13.015612 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b2c19-a95d-4542-b614-7cc3f53a780b None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:13.015895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b2c19-a95d-4542-b614-7cc3f53a780b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:13.023713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b2c19-a95d-4542-b614-7cc3f53a780b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:13.024393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-4938ef93-d986-4efb-a72f-77e055625130 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:28:13.024393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:13.024393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:28:13.024393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:28:13.024393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:28:13.024393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:28:13.024393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:28:13.024393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:13.024393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:28:13.024393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:13.024393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:28:13.024393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:13.024393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:13.024393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:13.024393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:13.024393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:13.024393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:13.025452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:13.025452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:13.025452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:13.025452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:13.025452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:13.025452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:13.025452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:28:13.025452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:28:13.025452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:28:13.025452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:28:13.025452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:13.025452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:13.025452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:13.025452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:13.025452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:13.025452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:13.026312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:13.026312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:13.026312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:13.026312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:13.026312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:13.026312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:13.026312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:13.026312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:13.026312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:13.026312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:13.026312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:28:13.026312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:13.026312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:28:13.026312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:13.027047 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b2c19-a95d-4542-b614-7cc3f53a780b None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:13.032674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 467/945] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 22:28:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:13.036353 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938ef93-d986-4efb-a72f-77e055625130 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:13.044873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-8c1b2c19-a95d-4542-b614-7cc3f53a780b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:28:13.044873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:13.044873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:13.044873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:13.044873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:13.044873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:13.044873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:13.044873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:13.044873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:13.044873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:13.044873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:13.044873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:13.044873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:28:13.044873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:28:13.044873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:28:13.044873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:28:13.045917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:13.045917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:13.045917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:28:13.045917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:28:13.045917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:28:13.045917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:28:13.045917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:13.045917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:13.045917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:13.045917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:13.045917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:13.045917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:13.045917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:13.045917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:13.045917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:13.045917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:13.045917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:13.045917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:13.046888 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:13.046888 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:13.046888 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:28:13.046888 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:28:13.046888 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:28:13.046888 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:28:13.046888 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:28:13.046888 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:28:13.046888 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:28:13.046888 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:13.051292 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 479/946] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 23 22:28:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:13.052882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b2c19-a95d-4542-b614-7cc3f53a780b None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:13.057160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:13.151487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938ef93-d986-4efb-a72f-77e055625130 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:13.154829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b2c19-a95d-4542-b614-7cc3f53a780b None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:13.189752 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30160c7-10fd-41f3-be60-b82ccf10f910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:13.190049 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad444f5f-cc29-40fe-8407-88f040db7e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:13.190586 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30160c7-10fd-41f3-be60-b82ccf10f910 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:13.190941 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30160c7-10fd-41f3-be60-b82ccf10f910 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:13.191293 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30160c7-10fd-41f3-be60-b82ccf10f910 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:13.191475 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad444f5f-cc29-40fe-8407-88f040db7e8b None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:13.191931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad444f5f-cc29-40fe-8407-88f040db7e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:13.192512 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30160c7-10fd-41f3-be60-b82ccf10f910 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:13.192778 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad444f5f-cc29-40fe-8407-88f040db7e8b None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:13.193066 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30160c7-10fd-41f3-be60-b82ccf10f910 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:13.193872 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad444f5f-cc29-40fe-8407-88f040db7e8b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:13.194361 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad444f5f-cc29-40fe-8407-88f040db7e8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:13.194695 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30160c7-10fd-41f3-be60-b82ccf10f910 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:13.196112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad444f5f-cc29-40fe-8407-88f040db7e8b None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:13.206502 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 468/947] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 23 22:28:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:13.208124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad444f5f-cc29-40fe-8407-88f040db7e8b None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:13.217823 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 480/948] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 23 22:28:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:13.218735 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30160c7-10fd-41f3-be60-b82ccf10f910 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:13.249544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:13.301600 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:13.325922 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30160c7-10fd-41f3-be60-b82ccf10f910 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:13.327666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad444f5f-cc29-40fe-8407-88f040db7e8b None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:13.363369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda03f0-5e55-4f86-98ae-e4b965c1e144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:13.364656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda03f0-5e55-4f86-98ae-e4b965c1e144 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:13.365108 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda03f0-5e55-4f86-98ae-e4b965c1e144 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:13.365461 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda03f0-5e55-4f86-98ae-e4b965c1e144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:13.367484 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df7045-6862-4b79-8e0b-ac2be5255c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:13.368352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df7045-6862-4b79-8e0b-ac2be5255c6a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:13.368860 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df7045-6862-4b79-8e0b-ac2be5255c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:13.369353 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df7045-6862-4b79-8e0b-ac2be5255c6a None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:13.370297 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df7045-6862-4b79-8e0b-ac2be5255c6a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:13.370798 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df7045-6862-4b79-8e0b-ac2be5255c6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:13.372408 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df7045-6862-4b79-8e0b-ac2be5255c6a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:13.376851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda03f0-5e55-4f86-98ae-e4b965c1e144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:13.382871 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda03f0-5e55-4f86-98ae-e4b965c1e144 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:13.383041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 469/949] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 23 22:28:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:13.383920 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df7045-6862-4b79-8e0b-ac2be5255c6a None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:13.404364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-5fda03f0-5e55-4f86-98ae-e4b965c1e144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:13.404364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:13.404364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:13.404364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:13.404364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:13.404364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:13.404364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:13.404364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:13.404364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:13.404364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:13.404364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:13.404364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:13.404364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:28:13.404364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:28:13.404364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:28:13.405406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:28:13.405406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:13.405406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:13.405406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:13.405406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:13.405406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:13.405406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:13.405406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:13.405406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:13.405406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:13.405406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:13.405406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:13.405406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:13.405406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:13.405406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:13.405406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:13.405406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:13.406294 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:28:13.406294 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:28:13.406294 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:13.406294 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:13.410441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 481/950] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:13.412185 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda03f0-5e55-4f86-98ae-e4b965c1e144 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:13.497414 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df7045-6862-4b79-8e0b-ac2be5255c6a None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:13.501044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:13.521558 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda03f0-5e55-4f86-98ae-e4b965c1e144 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:13.535871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfd215-916a-4a54-81dc-5176ee1ba04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:13.536952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfd215-916a-4a54-81dc-5176ee1ba04a None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:13.537463 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfd215-916a-4a54-81dc-5176ee1ba04a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:13.537946 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfd215-916a-4a54-81dc-5176ee1ba04a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:13.538914 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bfd215-916a-4a54-81dc-5176ee1ba04a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:13.540782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bfd215-916a-4a54-81dc-5176ee1ba04a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:13.551018 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:13.563354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47630b5-f5a9-4afa-965f-4bc303e6c1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:13.564026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47630b5-f5a9-4afa-965f-4bc303e6c1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:13.564317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47630b5-f5a9-4afa-965f-4bc303e6c1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:13.564641 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47630b5-f5a9-4afa-965f-4bc303e6c1f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:13.565357 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47630b5-f5a9-4afa-965f-4bc303e6c1f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:13.565655 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47630b5-f5a9-4afa-965f-4bc303e6c1f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:13.567634 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47630b5-f5a9-4afa-965f-4bc303e6c1f6 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:13.579807 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 482/951] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:13.585835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47630b5-f5a9-4afa-965f-4bc303e6c1f6 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:13.617270 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 470/952] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 23 22:28:13 2022] POST /identity/v3/endpoints => generated 349 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:28:13.709850 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47630b5-f5a9-4afa-965f-4bc303e6c1f6 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:13.759760 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ba9cd-53a9-4427-9808-a25ced8f7737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:13.760934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ba9cd-53a9-4427-9808-a25ced8f7737 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:13.761471 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ba9cd-53a9-4427-9808-a25ced8f7737 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:13.761955 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ba9cd-53a9-4427-9808-a25ced8f7737 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:13.765358 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:13.771837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ba9cd-53a9-4427-9808-a25ced8f7737 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:13.773733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ba9cd-53a9-4427-9808-a25ced8f7737 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:13.789595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-ce0ba9cd-53a9-4427-9808-a25ced8f7737 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:28:13.789595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:13.789595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:13.789595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:13.789595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:13.789595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:13.789595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:13.789595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:13.789595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:13.789595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:13.789595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:13.789595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:13.789595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:28:13.789595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:28:13.789595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:28:13.790831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:28:13.790831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:13.790831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:13.790831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:13.790831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:13.790831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:13.790831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:13.790831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:13.790831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:13.790831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:13.790831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:13.790831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:13.790831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:13.790831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:13.790831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:13.790831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:13.790831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:13.791777 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:28:13.791777 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:28:13.791777 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:28:13.791777 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:28:13.791777 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:28:13.791777 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:28:13.791777 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:28:13.791777 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:13.794022 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 483/953] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:28:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:13.800021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfd215-916a-4a54-81dc-5176ee1ba04a None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:13.884179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a2813-fd93-4c23-8c6b-57d50c27220b None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:13.884875 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a2813-fd93-4c23-8c6b-57d50c27220b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:13.885444 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a2813-fd93-4c23-8c6b-57d50c27220b None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:13.887355 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 484/954] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:28:13.893029 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d01ca-66db-4b85-998f-061e7f80975d None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:13.893327 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d01ca-66db-4b85-998f-061e7f80975d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:13.893616 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d01ca-66db-4b85-998f-061e7f80975d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:13.913057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-101d01ca-66db-4b85-998f-061e7f80975d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:13.914601 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-101d01ca-66db-4b85-998f-061e7f80975d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:13.915629 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfd215-916a-4a54-81dc-5176ee1ba04a None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:13.961728 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc54f6-26f1-47b0-a9ac-e7d41ac4ba2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:13.962775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc54f6-26f1-47b0-a9ac-e7d41ac4ba2c None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:13.963275 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc54f6-26f1-47b0-a9ac-e7d41ac4ba2c None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:13.964209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc54f6-26f1-47b0-a9ac-e7d41ac4ba2c None admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:13.965756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc54f6-26f1-47b0-a9ac-e7d41ac4ba2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:13.966563 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc54f6-26f1-47b0-a9ac-e7d41ac4ba2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:13.968469 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc54f6-26f1-47b0-a9ac-e7d41ac4ba2c None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:13.987687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 471/955] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:28:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:13.993974 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc54f6-26f1-47b0-a9ac-e7d41ac4ba2c None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:14.000686 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 485/956] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:28:14.017848 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d01ca-66db-4b85-998f-061e7f80975d None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:14.057479 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:14.105438 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc54f6-26f1-47b0-a9ac-e7d41ac4ba2c None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:14.124505 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d01ca-66db-4b85-998f-061e7f80975d None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:14.143279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22d76c-306a-4910-a7b5-cec59245bf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:14.144620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22d76c-306a-4910-a7b5-cec59245bf78 None admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:14.145698 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22d76c-306a-4910-a7b5-cec59245bf78 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:14.146203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22d76c-306a-4910-a7b5-cec59245bf78 None admin] PATH_INFO: `/v3/projects/14265887fea245a5a37abbb3a116e352/users/0a7c204a298c42f0bbc32fa9a6ce1f38/roles/b1cc8597c7b144ed9b6bed9fc40f3c97` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:14.149308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22d76c-306a-4910-a7b5-cec59245bf78 None admin] RBAC: Authorizing `identity:create_grant(project_id=14265887fea245a5a37abbb3a116e352, user_id=0a7c204a298c42f0bbc32fa9a6ce1f38, role_id=b1cc8597c7b144ed9b6bed9fc40f3c97)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:14.151152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22d76c-306a-4910-a7b5-cec59245bf78 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:14.153801 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:14.163268 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038f262-4763-4a47-b996-69c5f4b06490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:14.168862 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038f262-4763-4a47-b996-69c5f4b06490 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:14.168862 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038f262-4763-4a47-b996-69c5f4b06490 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:14.168862 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038f262-4763-4a47-b996-69c5f4b06490 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:14.168862 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038f262-4763-4a47-b996-69c5f4b06490 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:14.168862 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038f262-4763-4a47-b996-69c5f4b06490 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:14.178751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-4038f262-4763-4a47-b996-69c5f4b06490 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 22:28:14.178751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:14.178751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:28:14.178751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:28:14.178751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:28:14.178751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:28:14.178751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:28:14.178751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:14.178751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 22:28:14.178751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:14.178751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:28:14.178751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:14.178751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:14.178751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:14.178751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:14.178751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:14.178751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:14.179881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:14.179881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:14.179881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:14.179881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:14.179881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:14.179881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:14.179881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:28:14.179881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:28:14.179881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:28:14.179881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:28:14.179881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:14.179881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:14.179881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:14.179881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:14.179881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:14.179881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:14.180811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:14.180811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:14.180811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:14.180811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:14.180811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:14.180811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:14.180811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:14.180811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:14.180811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:14.180811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:14.180811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:28:14.180811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:14.180811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 22:28:14.180811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:14.180811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 486/957] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:28:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:14.185959 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038f262-4763-4a47-b996-69c5f4b06490 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:14.232659 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 472/958] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:28:13 2022] PUT /identity/v3/projects/14265887fea245a5a37abbb3a116e352/users/0a7c204a298c42f0bbc32fa9a6ce1f38/roles/b1cc8597c7b144ed9b6bed9fc40f3c97 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:28:14.320288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038f262-4763-4a47-b996-69c5f4b06490 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:14.360433 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f54eb-441c-4b65-a9d7-5a01aade20b0 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:14.360841 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f54eb-441c-4b65-a9d7-5a01aade20b0 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:14.361059 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f54eb-441c-4b65-a9d7-5a01aade20b0 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:14.361171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93b5c3-cb46-47d2-a7fc-fa6648fbc2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:14.361784 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93b5c3-cb46-47d2-a7fc-fa6648fbc2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:14.362075 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93b5c3-cb46-47d2-a7fc-fa6648fbc2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:14.362354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93b5c3-cb46-47d2-a7fc-fa6648fbc2b4 None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:14.362741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 473/959] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:28:14.363088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93b5c3-cb46-47d2-a7fc-fa6648fbc2b4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:14.363648 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93b5c3-cb46-47d2-a7fc-fa6648fbc2b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:14.368501 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93b5c3-cb46-47d2-a7fc-fa6648fbc2b4 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:14.370803 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08356be6-2b94-4e22-8ead-da3c2d2e616c None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:14.371329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08356be6-2b94-4e22-8ead-da3c2d2e616c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:14.372723 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 487/960] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 22:28:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:14.373293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08356be6-2b94-4e22-8ead-da3c2d2e616c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:14.377679 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93b5c3-cb46-47d2-a7fc-fa6648fbc2b4 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:14.397137 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-08356be6-2b94-4e22-8ead-da3c2d2e616c None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:14.398598 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-08356be6-2b94-4e22-8ead-da3c2d2e616c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:14.454604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:14.488738 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 474/961] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:28:14.504689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93b5c3-cb46-47d2-a7fc-fa6648fbc2b4 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:14.512427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08356be6-2b94-4e22-8ead-da3c2d2e616c None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:14.541926 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd67169-0d21-4fd1-96b0-c8e10714a101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:14.542907 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd67169-0d21-4fd1-96b0-c8e10714a101 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:14.543428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd67169-0d21-4fd1-96b0-c8e10714a101 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:14.543941 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd67169-0d21-4fd1-96b0-c8e10714a101 None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:14.544896 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd67169-0d21-4fd1-96b0-c8e10714a101 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:14.545428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd67169-0d21-4fd1-96b0-c8e10714a101 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:14.546997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd67169-0d21-4fd1-96b0-c8e10714a101 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:14.552923 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 488/962] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 22:28:14 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:14.568372 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:14.633127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08356be6-2b94-4e22-8ead-da3c2d2e616c None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:14.674526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec9800-623c-4ef4-8e4b-a3345ed70652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:14.675583 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec9800-623c-4ef4-8e4b-a3345ed70652 None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:14.676095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec9800-623c-4ef4-8e4b-a3345ed70652 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:14.676659 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec9800-623c-4ef4-8e4b-a3345ed70652 None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:14.677584 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec9800-623c-4ef4-8e4b-a3345ed70652 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:14.679279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec9800-623c-4ef4-8e4b-a3345ed70652 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:14.680942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: WARNING keystone.server.flask.common [None req-a4ec9800-623c-4ef4-8e4b-a3345ed70652 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 23 22:28:14.725776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.sql.core [None req-a4ec9800-623c-4ef4-8e4b-a3345ed70652 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 22:28:14.725776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 22:28:14.725776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [parameters: {'user_id': '6995ee9fb4b640c2ba208da92b002fdf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 22:28:14.725776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80519) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 22:28:14.734263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-a4ec9800-623c-4ef4-8e4b-a3345ed70652 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 22:28:14.734263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:14.734263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 22:28:14.734263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 22:28:14.734263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 22:28:14.734263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 22:28:14.734263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 22:28:14.734263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 22:28:14.734263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 22:28:14.734263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application conn.query(q) Jun 23 22:28:14.734263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 22:28:14.734263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 22:28:14.734263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 22:28:14.734263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application result.read() Jun 23 22:28:14.734263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 22:28:14.735190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 22:28:14.735190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 22:28:14.735190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 22:28:14.735190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 22:28:14.735190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 22:28:14.735190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 22:28:14.735190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 22:28:14.735190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 22:28:14.735190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:14.735190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 22:28:14.735190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:14.735190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:14.735190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 22:28:14.735190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 22:28:14.735190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 23 22:28:14.735190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 23 22:28:14.735190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 22:28:14.735190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application next(self.gen) Jun 23 22:28:14.736321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 22:28:14.736321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application yield resource Jun 23 22:28:14.736321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 22:28:14.736321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application next(self.gen) Jun 23 22:28:14.736321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 22:28:14.736321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 22:28:14.736321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 22:28:14.736321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 22:28:14.736321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 22:28:14.736321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise self.value Jun 23 22:28:14.736321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 22:28:14.736321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 22:28:14.736321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 22:28:14.736321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application session.commit() Jun 23 22:28:14.736321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 22:28:14.736321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 22:28:14.736321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 22:28:14.737256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 22:28:14.737256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 22:28:14.737256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self.session.flush() Jun 23 22:28:14.737256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 22:28:14.737256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 22:28:14.737256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 22:28:14.737256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 22:28:14.737256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 22:28:14.737256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application compat.raise_( Jun 23 22:28:14.737256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 22:28:14.737256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception Jun 23 22:28:14.737256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 22:28:14.737256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 22:28:14.737256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 22:28:14.737256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 22:28:14.737256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 22:28:14.737256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 22:28:14.738184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 22:28:14.738184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 22:28:14.738184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 23 22:28:14.738184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 23 22:28:14.738184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 22:28:14.738184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 22:28:14.738184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 22:28:14.738184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 22:28:14.738184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 22:28:14.738184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 22:28:14.738184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 22:28:14.738184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 22:28:14.738184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 22:28:14.738184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 22:28:14.738184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 22:28:14.738184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception Jun 23 22:28:14.739352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 22:28:14.739352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 22:28:14.739352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 22:28:14.739352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 22:28:14.739352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 22:28:14.739352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 22:28:14.739352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 22:28:14.739352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application conn.query(q) Jun 23 22:28:14.739352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 22:28:14.739352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 22:28:14.739352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 22:28:14.739352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application result.read() Jun 23 22:28:14.739352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 22:28:14.739352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 22:28:14.739352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 22:28:14.739352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 22:28:14.740276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 22:28:14.740276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 22:28:14.740276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 22:28:14.740276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 22:28:14.740276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 22:28:14.740276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 22:28:14.740276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [parameters: {'user_id': '6995ee9fb4b640c2ba208da92b002fdf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 22:28:14.740276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 22:28:14.740276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:14.740276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:28:14.740276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:14.740276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:14.740276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:14.740276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:14.740276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:14.742925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:14.742925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:14.742925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:14.742925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:14.742925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:14.742925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:14.742925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:14.742925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 23 22:28:14.742925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 23 22:28:14.742925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:14.742925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:14.742925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:28:14.742925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:28:14.742925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:28:14.742925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:28:14.742925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 23 22:28:14.742925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 23 22:28:14.745176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 23 22:28:14.745176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 23 22:28:14.745176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 22:28:14.745176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 22:28:14.745176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 22:28:14.745176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:14.747864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 475/963] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Jun 23 22:28:14 2022] POST /identity/v3/users => generated 160 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:14.754156 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd67169-0d21-4fd1-96b0-c8e10714a101 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:14.869603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd67169-0d21-4fd1-96b0-c8e10714a101 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:14.888116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:14.905293 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0361da2-b402-4e36-ace2-96f848468430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:14.906101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0361da2-b402-4e36-ace2-96f848468430 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:14.908696 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0361da2-b402-4e36-ace2-96f848468430 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:14.908696 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0361da2-b402-4e36-ace2-96f848468430 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:14.914652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0361da2-b402-4e36-ace2-96f848468430 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:14.916351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0361da2-b402-4e36-ace2-96f848468430 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:14.931186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-a0361da2-b402-4e36-ace2-96f848468430 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 22:28:14.931186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:14.931186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:14.931186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:14.931186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:14.931186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:14.931186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:14.931186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:14.931186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:14.931186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:14.931186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:14.931186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:14.931186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:28:14.931186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:28:14.931186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:28:14.931186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:28:14.932239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:14.932239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:14.932239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:28:14.932239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:28:14.932239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:28:14.932239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:28:14.932239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:14.932239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:14.932239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:14.932239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:14.932239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:14.932239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:14.932239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:14.932239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:14.932239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:14.932239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:14.932239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:14.932239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:14.933579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:14.933579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:14.933579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:28:14.933579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:28:14.933579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:28:14.933579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:28:14.933579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:28:14.933579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:28:14.933579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 22:28:14.933579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:14.933579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 489/964] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:28:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:14.940320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec9800-623c-4ef4-8e4b-a3345ed70652 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:15.050635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec9800-623c-4ef4-8e4b-a3345ed70652 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:15.061811 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:15.087231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2865b-2bad-4a74-9754-e18a144c2458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:15.088052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2865b-2bad-4a74-9754-e18a144c2458 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:15.088363 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2865b-2bad-4a74-9754-e18a144c2458 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:15.088746 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2865b-2bad-4a74-9754-e18a144c2458 None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:15.089636 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2865b-2bad-4a74-9754-e18a144c2458 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:15.090024 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2865b-2bad-4a74-9754-e18a144c2458 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:15.091724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2865b-2bad-4a74-9754-e18a144c2458 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:15.132125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 476/965] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 23 22:28:14 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:15.749858 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6220826-4d01-405b-b818-dc684f65c1a9 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:15.750422 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6220826-4d01-405b-b818-dc684f65c1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:15.750422 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6220826-4d01-405b-b818-dc684f65c1a9 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:15.752066 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 490/966] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:15.758191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b581484-409d-4c6f-b73f-093944654ab5 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:15.758694 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b581484-409d-4c6f-b73f-093944654ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:15.758850 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b581484-409d-4c6f-b73f-093944654ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:15.775897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-0b581484-409d-4c6f-b73f-093944654ab5 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:15.777454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-0b581484-409d-4c6f-b73f-093944654ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:15.848625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 477/967] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:15.869164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6220826-4d01-405b-b818-dc684f65c1a9 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:15.972105 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6220826-4d01-405b-b818-dc684f65c1a9 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:16.006244 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5d01b-baa7-40c7-8bd7-a218cdbd7ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:16.006973 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5d01b-baa7-40c7-8bd7-a218cdbd7ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:16.007256 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5d01b-baa7-40c7-8bd7-a218cdbd7ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:16.007531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5d01b-baa7-40c7-8bd7-a218cdbd7ba9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:16.008183 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5d01b-baa7-40c7-8bd7-a218cdbd7ba9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:16.009578 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5d01b-baa7-40c7-8bd7-a218cdbd7ba9 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:16.022298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-1ec5d01b-baa7-40c7-8bd7-a218cdbd7ba9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:28:16.022298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:16.022298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:28:16.022298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:28:16.022298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:28:16.022298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:28:16.022298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:28:16.022298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:16.022298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:28:16.022298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:16.022298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:28:16.022298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:16.022298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:16.022298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:16.022298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:16.022298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:16.022298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:16.023330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:16.023330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:16.023330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:16.023330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:16.023330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:16.023330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:16.023330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:28:16.023330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:28:16.023330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:28:16.023330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:28:16.023330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:16.023330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:16.023330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:16.023330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:16.023330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:16.023330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:16.024199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:16.024199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:16.024199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:16.024199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:16.024199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:16.024199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:16.024199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:16.024199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:16.024199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:16.024199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:16.024199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:28:16.024199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:16.024199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:28:16.024199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:16.031472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 491/968] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 22:28:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:16.031602 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b581484-409d-4c6f-b73f-093944654ab5 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:16.125433 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:16.131564 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b581484-409d-4c6f-b73f-093944654ab5 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:16.165897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546392c-584d-4023-807b-1358a1c09cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:16.166692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546392c-584d-4023-807b-1358a1c09cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:16.167045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546392c-584d-4023-807b-1358a1c09cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:16.167386 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546392c-584d-4023-807b-1358a1c09cc0 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:16.168155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546392c-584d-4023-807b-1358a1c09cc0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:16.168531 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546392c-584d-4023-807b-1358a1c09cc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:16.170244 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546392c-584d-4023-807b-1358a1c09cc0 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:16.180117 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 478/969] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 23 22:28:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:16.181629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5d01b-baa7-40c7-8bd7-a218cdbd7ba9 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:16.226093 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:16.293339 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5d01b-baa7-40c7-8bd7-a218cdbd7ba9 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:16.326737 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d2c6b-fb00-4da7-a224-9a787301931a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:16.327494 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d2c6b-fb00-4da7-a224-9a787301931a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:16.327844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d2c6b-fb00-4da7-a224-9a787301931a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:16.330999 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d2c6b-fb00-4da7-a224-9a787301931a None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:16.330999 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d2c6b-fb00-4da7-a224-9a787301931a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:16.330999 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d2c6b-fb00-4da7-a224-9a787301931a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:16.330999 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d2c6b-fb00-4da7-a224-9a787301931a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:16.341076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 492/970] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 23 22:28:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:16.342157 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546392c-584d-4023-807b-1358a1c09cc0 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:16.423447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce956dfa-b3ab-4645-8e2a-737ddd9ed83d None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:16.423755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce956dfa-b3ab-4645-8e2a-737ddd9ed83d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:16.424013 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce956dfa-b3ab-4645-8e2a-737ddd9ed83d None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:16.425643 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 493/971] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:16.430492 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cd527-4b11-4704-8341-7b682d5efde6 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:16.430781 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cd527-4b11-4704-8341-7b682d5efde6 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:16.431061 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cd527-4b11-4704-8341-7b682d5efde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:16.444020 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546392c-584d-4023-807b-1358a1c09cc0 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:16.448260 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-398cd527-4b11-4704-8341-7b682d5efde6 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:16.449472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-398cd527-4b11-4704-8341-7b682d5efde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:16.480615 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa9731-9a3a-4ad0-a1e9-2e333c340271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:16.481274 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa9731-9a3a-4ad0-a1e9-2e333c340271 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:16.481558 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa9731-9a3a-4ad0-a1e9-2e333c340271 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:16.481835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa9731-9a3a-4ad0-a1e9-2e333c340271 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:16.482628 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa9731-9a3a-4ad0-a1e9-2e333c340271 None admin] RBAC: Adding query filter params (interface=admin, service_id=39274e290c77462ba56e8f8e776fd4e9, region_id=RegionOne) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:16.482941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa9731-9a3a-4ad0-a1e9-2e333c340271 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:16.484311 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa9731-9a3a-4ad0-a1e9-2e333c340271 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:16.490574 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 479/972] 10.222.0.33 () {56 vars in 1249 bytes} [Thu Jun 23 22:28:16 2022] GET /identity/v3/endpoints?service_id=39274e290c77462ba56e8f8e776fd4e9&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:16.495645 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa9731-9a3a-4ad0-a1e9-2e333c340271 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:16.519194 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 494/973] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:28:16.522488 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:16.544545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cd527-4b11-4704-8341-7b682d5efde6 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:16.580242 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:16.605043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa9731-9a3a-4ad0-a1e9-2e333c340271 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:16.642309 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add74b10-35ff-4eee-b7f6-48ee5e8cee0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:16.642967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add74b10-35ff-4eee-b7f6-48ee5e8cee0a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:16.643276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add74b10-35ff-4eee-b7f6-48ee5e8cee0a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:16.643522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add74b10-35ff-4eee-b7f6-48ee5e8cee0a None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:16.644196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add74b10-35ff-4eee-b7f6-48ee5e8cee0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:16.646349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add74b10-35ff-4eee-b7f6-48ee5e8cee0a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:16.647682 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cd527-4b11-4704-8341-7b682d5efde6 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:16.652684 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 480/974] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 23 22:28:16 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:28:16.686797 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43e329-0fb0-4713-ae00-cbb42c0fcbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:16.687634 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43e329-0fb0-4713-ae00-cbb42c0fcbc4 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:16.687918 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43e329-0fb0-4713-ae00-cbb42c0fcbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:16.688192 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43e329-0fb0-4713-ae00-cbb42c0fcbc4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:16.700829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43e329-0fb0-4713-ae00-cbb42c0fcbc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:28:16.709829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-7b43e329-0fb0-4713-ae00-cbb42c0fcbc4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:28:16.709829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:16.709829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:16.709829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:16.709829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:16.709829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:16.709829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:16.709829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:16.709829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:16.709829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:16.709829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:16.709829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:16.709829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:28:16.709829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:28:16.709829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:28:16.709829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:28:16.710873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:28:16.710873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:28:16.710873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:28:16.710873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:28:16.710873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:16.710873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:16.710873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:16.710873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:16.710873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:16.710873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:16.710873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:16.710873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:16.710873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:16.710873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:16.710873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:16.710873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:16.711717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:16.711717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:16.711717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:16.711717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:16.711717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:28:16.711717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:28:16.711717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:28:16.711717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:28:16.711717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:28:16.711717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:28:16.711717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:28:16.711717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:16.716010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 495/975] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 23 22:28:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:16.716871 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43e329-0fb0-4713-ae00-cbb42c0fcbc4 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:16.717401 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa07bf-ccc2-4482-ac8b-428b5460898a None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:16.717747 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa07bf-ccc2-4482-ac8b-428b5460898a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:16.717961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa07bf-ccc2-4482-ac8b-428b5460898a None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:16.719532 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 481/976] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:28:16.725452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f92be-e69c-44a8-887c-4529d32082ec None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:16.725755 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f92be-e69c-44a8-887c-4529d32082ec None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:16.726027 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f92be-e69c-44a8-887c-4529d32082ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:16.743664 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-c91f92be-e69c-44a8-887c-4529d32082ec None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:16.744912 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c91f92be-e69c-44a8-887c-4529d32082ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:16.822126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 482/977] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:28:16.836860 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:16.843439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43e329-0fb0-4713-ae00-cbb42c0fcbc4 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:16.847195 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f92be-e69c-44a8-887c-4529d32082ec None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:16.868940 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:16.880102 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21397c39-a391-414a-b26d-36fedcd2e223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:16.880909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21397c39-a391-414a-b26d-36fedcd2e223 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:16.881260 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21397c39-a391-414a-b26d-36fedcd2e223 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:16.881601 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21397c39-a391-414a-b26d-36fedcd2e223 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:16.882469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21397c39-a391-414a-b26d-36fedcd2e223 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:16.882839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21397c39-a391-414a-b26d-36fedcd2e223 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:16.884334 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21397c39-a391-414a-b26d-36fedcd2e223 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:16.894979 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 496/978] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 23 22:28:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:16.900994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21397c39-a391-414a-b26d-36fedcd2e223 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:16.961031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f92be-e69c-44a8-887c-4529d32082ec None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:17.003378 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071621dd-36a3-4bbd-b57c-99e7d833b371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:17.004845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071621dd-36a3-4bbd-b57c-99e7d833b371 None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:17.005353 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071621dd-36a3-4bbd-b57c-99e7d833b371 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:17.005633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071621dd-36a3-4bbd-b57c-99e7d833b371 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:17.006365 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071621dd-36a3-4bbd-b57c-99e7d833b371 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:17.007747 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071621dd-36a3-4bbd-b57c-99e7d833b371 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:17.008766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21397c39-a391-414a-b26d-36fedcd2e223 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:17.043193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e51561-289c-4df9-ae35-2b7249554024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:17.043989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e51561-289c-4df9-ae35-2b7249554024 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:17.044274 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e51561-289c-4df9-ae35-2b7249554024 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:17.044551 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e51561-289c-4df9-ae35-2b7249554024 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:17.055135 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e51561-289c-4df9-ae35-2b7249554024 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:17.056560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e51561-289c-4df9-ae35-2b7249554024 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:17.074088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-71e51561-289c-4df9-ae35-2b7249554024 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:17.074088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:17.074088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:17.074088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:17.074088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:17.074088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:17.074088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:17.074088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:17.074088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:17.074088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:17.074088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:17.074088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:17.074088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:28:17.074088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:28:17.074088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:28:17.075186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:28:17.075186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:17.075186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:17.075186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:17.075186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:17.075186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:17.075186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:17.075186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:17.075186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:17.075186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:17.075186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:17.075186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:17.075186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:17.075186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:17.075186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:17.075186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:17.075186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:17.081124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:28:17.081124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:28:17.081124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:17.081124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:17.081124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 497/979] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:17.081124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e51561-289c-4df9-ae35-2b7249554024 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:17.087645 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 483/980] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 22:28:16 2022] POST /identity/v3/services => generated 242 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:28:17.095381 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:17.194276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e51561-289c-4df9-ae35-2b7249554024 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:17.237704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd2ba4-6e9f-41af-a357-0c33332c0fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:17.238381 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd2ba4-6e9f-41af-a357-0c33332c0fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:17.238697 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd2ba4-6e9f-41af-a357-0c33332c0fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:17.238982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd2ba4-6e9f-41af-a357-0c33332c0fe5 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:17.239906 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd2ba4-6e9f-41af-a357-0c33332c0fe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:17.240217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd2ba4-6e9f-41af-a357-0c33332c0fe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:17.241624 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd2ba4-6e9f-41af-a357-0c33332c0fe5 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:17.259806 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 498/981] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:17.261187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071621dd-36a3-4bbd-b57c-99e7d833b371 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:17.348639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:17.369026 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071621dd-36a3-4bbd-b57c-99e7d833b371 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:17.402809 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a660e0-edf5-40a3-88cf-088917a61aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:17.403624 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a660e0-edf5-40a3-88cf-088917a61aee None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:17.403911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a660e0-edf5-40a3-88cf-088917a61aee None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:17.404185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a660e0-edf5-40a3-88cf-088917a61aee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:17.411962 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a660e0-edf5-40a3-88cf-088917a61aee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:17.413444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a660e0-edf5-40a3-88cf-088917a61aee None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:17.428708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-43a660e0-edf5-40a3-88cf-088917a61aee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:28:17.428708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:17.428708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:17.428708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:17.428708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:17.428708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:17.428708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:17.428708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:17.428708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:17.428708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:17.428708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:17.428708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:17.428708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:28:17.428708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:28:17.428708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:28:17.428708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:28:17.429710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:17.429710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:17.429710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:28:17.429710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:28:17.429710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:28:17.429710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:28:17.429710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:17.429710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:17.429710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:17.429710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:17.429710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:17.429710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:17.429710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:17.429710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:17.429710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:17.429710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:17.429710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:17.429710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:17.435444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:17.435444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:17.435444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:28:17.435444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:28:17.435444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:28:17.435444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:28:17.435444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:28:17.435444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:28:17.435444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:28:17.435444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:17.435444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 484/982] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 23 22:28:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:17.438617 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd2ba4-6e9f-41af-a357-0c33332c0fe5 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:17.506376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:17.541842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd2ba4-6e9f-41af-a357-0c33332c0fe5 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:17.578118 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c82ee-1f84-42be-a1df-fa15538c5776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:17.578876 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c82ee-1f84-42be-a1df-fa15538c5776 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:17.579227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c82ee-1f84-42be-a1df-fa15538c5776 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:17.579567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c82ee-1f84-42be-a1df-fa15538c5776 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:17.580472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c82ee-1f84-42be-a1df-fa15538c5776 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:17.580880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c82ee-1f84-42be-a1df-fa15538c5776 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:17.582369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c82ee-1f84-42be-a1df-fa15538c5776 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:17.600662 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 499/983] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 23 22:28:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:17.606038 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a660e0-edf5-40a3-88cf-088917a61aee None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:17.705678 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a660e0-edf5-40a3-88cf-088917a61aee None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:17.739801 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4f596-6d5b-4f2f-b558-86ec4a42b329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:17.740452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4f596-6d5b-4f2f-b558-86ec4a42b329 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:17.740765 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4f596-6d5b-4f2f-b558-86ec4a42b329 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:17.741042 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4f596-6d5b-4f2f-b558-86ec4a42b329 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:17.749853 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:17.812370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4f596-6d5b-4f2f-b558-86ec4a42b329 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:17.813957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4f596-6d5b-4f2f-b558-86ec4a42b329 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:17.918288 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 485/984] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:28:17 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:28:17.920491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c82ee-1f84-42be-a1df-fa15538c5776 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:18.017583 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c82ee-1f84-42be-a1df-fa15538c5776 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:18.041254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:18.052367 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c1074-5b89-4493-8234-4c62e2c73310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:18.053310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c1074-5b89-4493-8234-4c62e2c73310 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:18.053594 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c1074-5b89-4493-8234-4c62e2c73310 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:18.053980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c1074-5b89-4493-8234-4c62e2c73310 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:18.063853 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c1074-5b89-4493-8234-4c62e2c73310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:18.065330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c1074-5b89-4493-8234-4c62e2c73310 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:18.083269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-510c1074-5b89-4493-8234-4c62e2c73310 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:18.083269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:18.083269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:18.083269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:18.083269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:18.083269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:18.083269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:18.083269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:18.083269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:18.083269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:18.083269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:18.083269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:18.083269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:28:18.083269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:28:18.083269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:28:18.084160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:28:18.084160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:18.084160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:18.084160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:18.084160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:18.084160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:18.084160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:18.084160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:18.084160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:18.084160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:18.084160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:18.084160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:18.084160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:18.084160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:18.084160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:18.084160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:18.084160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:18.089814 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:28:18.089814 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:28:18.089814 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:18.089814 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:18.089814 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 500/985] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:18.091463 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4f596-6d5b-4f2f-b558-86ec4a42b329 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:18.132084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:18.197395 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4f596-6d5b-4f2f-b558-86ec4a42b329 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:18.234288 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f3484-567a-463a-9fd7-604a37cd4d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:18.234954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f3484-567a-463a-9fd7-604a37cd4d00 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:18.235235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f3484-567a-463a-9fd7-604a37cd4d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:18.235512 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f3484-567a-463a-9fd7-604a37cd4d00 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:18.236239 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f3484-567a-463a-9fd7-604a37cd4d00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:18.236544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f3484-567a-463a-9fd7-604a37cd4d00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:18.237943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f3484-567a-463a-9fd7-604a37cd4d00 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:18.249711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 486/986] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:18.256211 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c1074-5b89-4493-8234-4c62e2c73310 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:18.364346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c1074-5b89-4493-8234-4c62e2c73310 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:18.460231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafc955-9846-403c-a010-057c68598e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:18.461393 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafc955-9846-403c-a010-057c68598e5a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:18.461901 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafc955-9846-403c-a010-057c68598e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:18.462450 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafc955-9846-403c-a010-057c68598e5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:18.469380 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aafc955-9846-403c-a010-057c68598e5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:18.471025 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aafc955-9846-403c-a010-057c68598e5a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:18.484981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-9aafc955-9846-403c-a010-057c68598e5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:28:18.484981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:18.484981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:18.484981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:18.484981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:18.484981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:18.484981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:18.484981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:18.484981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:18.484981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:18.484981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:18.484981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:18.484981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:28:18.484981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:28:18.484981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:28:18.485940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:28:18.485940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:18.485940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:18.485940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:18.485940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:18.485940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:18.485940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:18.485940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:18.485940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:18.485940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:18.485940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:18.485940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:18.485940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:18.485940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:18.485940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:18.485940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:18.485940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:18.486813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:28:18.486813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:28:18.486813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:28:18.486813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:28:18.486813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:28:18.486813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:28:18.486813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:28:18.486813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:18.489778 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 501/987] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:28:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:18.496880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f3484-567a-463a-9fd7-604a37cd4d00 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:18.594384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:18.598691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f3484-567a-463a-9fd7-604a37cd4d00 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:18.634256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b43133-1cc3-4726-aa03-7744e51228d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:18.634914 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b43133-1cc3-4726-aa03-7744e51228d7 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:18.635196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b43133-1cc3-4726-aa03-7744e51228d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:18.635473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b43133-1cc3-4726-aa03-7744e51228d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:18.636250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b43133-1cc3-4726-aa03-7744e51228d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:18.636560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b43133-1cc3-4726-aa03-7744e51228d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:18.638018 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b43133-1cc3-4726-aa03-7744e51228d7 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:18.654287 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 487/988] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:28:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:18.655942 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafc955-9846-403c-a010-057c68598e5a None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:18.694512 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:18.724963 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24881884-75bc-4371-b81a-ef834994357f None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:18.725257 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24881884-75bc-4371-b81a-ef834994357f None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:18.725447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24881884-75bc-4371-b81a-ef834994357f None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:18.726837 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 488/989] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:18.731698 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c3548-9b64-4d3b-a50f-ecdee5e002bf None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:18.731990 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c3548-9b64-4d3b-a50f-ecdee5e002bf None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:18.732274 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c3548-9b64-4d3b-a50f-ecdee5e002bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:18.749086 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-6b4c3548-9b64-4d3b-a50f-ecdee5e002bf None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:18.750274 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-6b4c3548-9b64-4d3b-a50f-ecdee5e002bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:18.765396 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafc955-9846-403c-a010-057c68598e5a None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:18.800439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6ff02-2b2d-43db-bb8f-25ceff4027c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:18.801759 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6ff02-2b2d-43db-bb8f-25ceff4027c1 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:18.802067 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6ff02-2b2d-43db-bb8f-25ceff4027c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:18.802340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6ff02-2b2d-43db-bb8f-25ceff4027c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:18.803201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6ff02-2b2d-43db-bb8f-25ceff4027c1 None admin] RBAC: Adding query filter params (role.id=b1cc8597c7b144ed9b6bed9fc40f3c97, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=0a7c204a298c42f0bbc32fa9a6ce1f38) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:18.803504 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6ff02-2b2d-43db-bb8f-25ceff4027c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:18.804963 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6ff02-2b2d-43db-bb8f-25ceff4027c1 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:18.810835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 502/990] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:28:18 2022] GET /identity/v3/role_assignments?user.id=0a7c204a298c42f0bbc32fa9a6ce1f38&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=b1cc8597c7b144ed9b6bed9fc40f3c97 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:18.821125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 489/991] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:28:18.844943 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6ff02-2b2d-43db-bb8f-25ceff4027c1 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:18.954147 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:18.957724 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6ff02-2b2d-43db-bb8f-25ceff4027c1 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:18.993767 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2a553-a74a-41bc-ab73-8faa62eee7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:18.994582 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2a553-a74a-41bc-ab73-8faa62eee7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:18.994863 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2a553-a74a-41bc-ab73-8faa62eee7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:18.995135 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2a553-a74a-41bc-ab73-8faa62eee7f3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:18.995789 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2a553-a74a-41bc-ab73-8faa62eee7f3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:18.997181 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2a553-a74a-41bc-ab73-8faa62eee7f3 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:19.009811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-b9c2a553-a74a-41bc-ab73-8faa62eee7f3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:28:19.009811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:19.009811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:28:19.009811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:28:19.009811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:28:19.009811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:28:19.009811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:28:19.009811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:19.009811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:28:19.009811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:19.009811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:28:19.009811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:19.009811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:19.009811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:19.009811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:19.009811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:19.009811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:19.011040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:19.011040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:19.011040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:19.011040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:19.011040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:19.011040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:19.011040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:28:19.011040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:28:19.011040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:28:19.011040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:28:19.011040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:19.011040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:19.011040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:19.011040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:19.011040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:19.011040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:19.012104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:19.012104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:19.012104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:19.012104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:19.012104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:19.012104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:19.012104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:19.012104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:19.012104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:19.012104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:19.012104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:28:19.012104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:19.012104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:28:19.012104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:19.012104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 503/992] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 23 22:28:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:19.018469 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c3548-9b64-4d3b-a50f-ecdee5e002bf None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:19.106872 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:19.119459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c3548-9b64-4d3b-a50f-ecdee5e002bf None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:19.154062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b60b3-86cf-4e6b-add2-78ca639d1e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:19.154727 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b60b3-86cf-4e6b-add2-78ca639d1e78 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:19.155010 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b60b3-86cf-4e6b-add2-78ca639d1e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:19.155293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b60b3-86cf-4e6b-add2-78ca639d1e78 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:19.155984 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b60b3-86cf-4e6b-add2-78ca639d1e78 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:19.156290 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b60b3-86cf-4e6b-add2-78ca639d1e78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:19.157681 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b60b3-86cf-4e6b-add2-78ca639d1e78 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:19.168278 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 490/993] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 23 22:28:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:19.169766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2a553-a74a-41bc-ab73-8faa62eee7f3 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:19.170878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de765ad2-3d2a-4a38-b35e-627592572960 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:19.171178 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de765ad2-3d2a-4a38-b35e-627592572960 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:19.171463 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de765ad2-3d2a-4a38-b35e-627592572960 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:19.172833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 491/994] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:28:19.179025 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe530e-abb1-4236-b3a4-972654a971bc None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:19.179518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe530e-abb1-4236-b3a4-972654a971bc None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:19.180418 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe530e-abb1-4236-b3a4-972654a971bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:19.201679 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-0ffe530e-abb1-4236-b3a4-972654a971bc None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:19.203618 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-0ffe530e-abb1-4236-b3a4-972654a971bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:19.280733 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 492/995] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:28:19.281089 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2a553-a74a-41bc-ab73-8faa62eee7f3 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:19.304500 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe530e-abb1-4236-b3a4-972654a971bc None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:19.323935 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc97051-bc58-4cec-9ba7-1af19fbc87a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:19.324633 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc97051-bc58-4cec-9ba7-1af19fbc87a9 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:19.324921 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc97051-bc58-4cec-9ba7-1af19fbc87a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:19.325207 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc97051-bc58-4cec-9ba7-1af19fbc87a9 None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:19.325918 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc97051-bc58-4cec-9ba7-1af19fbc87a9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:19.326230 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc97051-bc58-4cec-9ba7-1af19fbc87a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:19.327605 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc97051-bc58-4cec-9ba7-1af19fbc87a9 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:19.335087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 504/996] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 23 22:28:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:19.340256 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc97051-bc58-4cec-9ba7-1af19fbc87a9 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:19.373214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:19.397713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:19.418383 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe530e-abb1-4236-b3a4-972654a971bc None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:19.449534 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc97051-bc58-4cec-9ba7-1af19fbc87a9 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:19.452911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d036f0c-7e20-45dd-b7da-2ef812b004dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:19.453716 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d036f0c-7e20-45dd-b7da-2ef812b004dc None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:19.454262 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d036f0c-7e20-45dd-b7da-2ef812b004dc None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:19.454598 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d036f0c-7e20-45dd-b7da-2ef812b004dc None admin] PATH_INFO: `/v3/services/compute` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:19.455589 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d036f0c-7e20-45dd-b7da-2ef812b004dc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:19.458126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d036f0c-7e20-45dd-b7da-2ef812b004dc None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:19.470177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-3d036f0c-7e20-45dd-b7da-2ef812b004dc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 22:28:19.470177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:19.470177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:28:19.470177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:28:19.470177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:28:19.470177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:28:19.470177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:28:19.470177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:19.470177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 22:28:19.470177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:19.470177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:28:19.470177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:19.470177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:19.470177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:19.470177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:19.470177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:19.470177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:19.471191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:19.471191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:19.471191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:19.471191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:19.471191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:19.471191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:19.471191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:28:19.471191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:28:19.471191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:28:19.471191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:28:19.471191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:19.471191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:19.471191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:19.471191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:19.471191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:19.471191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:19.474416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:19.474416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:19.474416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:19.474416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:19.474416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:19.474416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:19.474416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:19.474416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:19.474416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:19.474416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:19.474416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:28:19.474416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:19.474416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 22:28:19.474416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:19.474416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 493/997] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:28:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:19.478089 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d036f0c-7e20-45dd-b7da-2ef812b004dc None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:19.490955 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a607d5-04b4-4c25-adb7-9d058703df47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:19.491623 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a607d5-04b4-4c25-adb7-9d058703df47 None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:19.491908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a607d5-04b4-4c25-adb7-9d058703df47 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:19.492185 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a607d5-04b4-4c25-adb7-9d058703df47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:19.492957 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a607d5-04b4-4c25-adb7-9d058703df47 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:19.494318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a607d5-04b4-4c25-adb7-9d058703df47 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:19.553900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 505/998] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 23 22:28:19 2022] POST /identity/v3/endpoints => generated 325 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:28:19.593840 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d036f0c-7e20-45dd-b7da-2ef812b004dc None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:19.641067 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c5e95-5ef0-41b4-b013-848a7fc53592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:19.642084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c5e95-5ef0-41b4-b013-848a7fc53592 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:19.642690 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c5e95-5ef0-41b4-b013-848a7fc53592 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:19.643145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c5e95-5ef0-41b4-b013-848a7fc53592 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:19.644039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c5e95-5ef0-41b4-b013-848a7fc53592 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:19.644537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c5e95-5ef0-41b4-b013-848a7fc53592 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:19.646607 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c5e95-5ef0-41b4-b013-848a7fc53592 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:19.657011 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 494/999] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 22:28:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:19.658983 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a607d5-04b4-4c25-adb7-9d058703df47 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:19.755601 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:19.768338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a607d5-04b4-4c25-adb7-9d058703df47 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:19.802908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54422cf3-3536-4b58-83dd-7f5c5729fb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:19.803896 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54422cf3-3536-4b58-83dd-7f5c5729fb76 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:19.804429 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54422cf3-3536-4b58-83dd-7f5c5729fb76 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:19.804888 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54422cf3-3536-4b58-83dd-7f5c5729fb76 None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:19.805741 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54422cf3-3536-4b58-83dd-7f5c5729fb76 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:19.806214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54422cf3-3536-4b58-83dd-7f5c5729fb76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:19.807739 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54422cf3-3536-4b58-83dd-7f5c5729fb76 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:19.817801 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 506/1000] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 22:28:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:19.819594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c5e95-5ef0-41b4-b013-848a7fc53592 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:19.829785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:19.921914 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c5e95-5ef0-41b4-b013-848a7fc53592 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:19.957049 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a1d7a-cd30-4fd7-8c4b-06dafbef3cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:19.958009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a1d7a-cd30-4fd7-8c4b-06dafbef3cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:19.958446 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a1d7a-cd30-4fd7-8c4b-06dafbef3cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:19.958874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a1d7a-cd30-4fd7-8c4b-06dafbef3cd4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:19.959808 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a1d7a-cd30-4fd7-8c4b-06dafbef3cd4 None admin] RBAC: Adding query filter params (interface=public, service_id=e4734597fd004bd187c6bc1337ea7056, region_id=RegionOne) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:19.960272 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a1d7a-cd30-4fd7-8c4b-06dafbef3cd4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:19.961833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a1d7a-cd30-4fd7-8c4b-06dafbef3cd4 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:19.967200 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 495/1001] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 23 22:28:19 2022] GET /identity/v3/endpoints?service_id=e4734597fd004bd187c6bc1337ea7056&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:19.973010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54422cf3-3536-4b58-83dd-7f5c5729fb76 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:20.078899 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54422cf3-3536-4b58-83dd-7f5c5729fb76 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:20.111682 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf1bcf-c26f-4105-ba06-46b03d58263d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:20.112357 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf1bcf-c26f-4105-ba06-46b03d58263d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:20.112669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf1bcf-c26f-4105-ba06-46b03d58263d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:20.112949 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf1bcf-c26f-4105-ba06-46b03d58263d None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:20.114192 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf1bcf-c26f-4105-ba06-46b03d58263d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:20.115585 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf1bcf-c26f-4105-ba06-46b03d58263d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:20.128611 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 507/1002] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 23 22:28:19 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:28:20.905555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc223f-c058-4304-8c9c-bab1dde44826 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:20.906077 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc223f-c058-4304-8c9c-bab1dde44826 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:20.906156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc223f-c058-4304-8c9c-bab1dde44826 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:20.907611 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 496/1003] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:20.913134 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67436b6a-2016-444f-b90b-35de55bd7fab None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:20.913429 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67436b6a-2016-444f-b90b-35de55bd7fab None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:20.913711 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67436b6a-2016-444f-b90b-35de55bd7fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:20.930304 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-67436b6a-2016-444f-b90b-35de55bd7fab None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:20.931473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-67436b6a-2016-444f-b90b-35de55bd7fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:20.942530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:20.998518 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 508/1004] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:20 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:21.021416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccc223f-c058-4304-8c9c-bab1dde44826 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:21.094286 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:21.126256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccc223f-c058-4304-8c9c-bab1dde44826 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:21.160897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0602461-7d31-475a-b055-ca4b40943336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:21.161628 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0602461-7d31-475a-b055-ca4b40943336 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:21.161918 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0602461-7d31-475a-b055-ca4b40943336 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:21.162195 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0602461-7d31-475a-b055-ca4b40943336 None admin] PATH_INFO: `/v3/services/image` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:21.162846 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0602461-7d31-475a-b055-ca4b40943336 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:21.164218 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0602461-7d31-475a-b055-ca4b40943336 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:21.176002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-d0602461-7d31-475a-b055-ca4b40943336 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 22:28:21.176002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:21.176002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:28:21.176002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:28:21.176002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:28:21.176002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:28:21.176002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:28:21.176002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:21.176002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 22:28:21.176002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:21.176002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:28:21.176002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:21.176002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:21.176002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:21.176002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:21.176002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:21.176002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:21.177225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:21.177225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:21.177225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:21.177225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:21.177225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:21.177225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:21.177225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:28:21.177225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:28:21.177225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:28:21.177225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:28:21.177225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:21.177225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:21.177225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:21.177225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:21.177225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:21.177225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:21.178081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:21.178081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:21.178081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:21.178081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:21.178081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:21.178081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:21.178081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:21.178081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:21.178081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:21.178081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:21.178081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:28:21.178081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:21.178081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 22:28:21.178081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:21.178081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 497/1005] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Jun 23 22:28:21 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:21.185089 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67436b6a-2016-444f-b90b-35de55bd7fab None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:21.299481 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67436b6a-2016-444f-b90b-35de55bd7fab None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:21.335392 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1340d5-65ad-4bd8-ad01-0ad7ab2b7e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:21.336057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1340d5-65ad-4bd8-ad01-0ad7ab2b7e6b None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:21.336347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1340d5-65ad-4bd8-ad01-0ad7ab2b7e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:21.336660 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1340d5-65ad-4bd8-ad01-0ad7ab2b7e6b None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:21.337368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1340d5-65ad-4bd8-ad01-0ad7ab2b7e6b None admin] RBAC: Adding query filter params (name=image) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:21.337675 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1340d5-65ad-4bd8-ad01-0ad7ab2b7e6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:21.339039 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1340d5-65ad-4bd8-ad01-0ad7ab2b7e6b None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:21.348689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 509/1006] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:21.349735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0602461-7d31-475a-b055-ca4b40943336 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:21.475340 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0602461-7d31-475a-b055-ca4b40943336 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:21.507823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab8d20-3995-416b-a682-a5d7722b8cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:21.508479 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab8d20-3995-416b-a682-a5d7722b8cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:21.508855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab8d20-3995-416b-a682-a5d7722b8cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:21.509140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab8d20-3995-416b-a682-a5d7722b8cd3 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:21.509840 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab8d20-3995-416b-a682-a5d7722b8cd3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:21.510147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab8d20-3995-416b-a682-a5d7722b8cd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:21.511520 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab8d20-3995-416b-a682-a5d7722b8cd3 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:21.512829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:21.524879 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 498/1007] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:21.617867 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a7ae9-514b-4f85-97b7-cc2496f4df47 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:21.618162 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a7ae9-514b-4f85-97b7-cc2496f4df47 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:21.618367 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a7ae9-514b-4f85-97b7-cc2496f4df47 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:21.619883 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 510/1008] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:21.625517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635f2c5-f3eb-4151-af9e-8607611ff758 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:21.625826 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635f2c5-f3eb-4151-af9e-8607611ff758 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:21.626113 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635f2c5-f3eb-4151-af9e-8607611ff758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:21.643521 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-b635f2c5-f3eb-4151-af9e-8607611ff758 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:21.645319 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-b635f2c5-f3eb-4151-af9e-8607611ff758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:21.715946 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 499/1009] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:21 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:21.739847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a7ae9-514b-4f85-97b7-cc2496f4df47 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:21.752475 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:21.843907 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a7ae9-514b-4f85-97b7-cc2496f4df47 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:21.880249 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad554dae-060c-4f52-85d2-c38016bfc9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:21.881094 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad554dae-060c-4f52-85d2-c38016bfc9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:21.881380 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad554dae-060c-4f52-85d2-c38016bfc9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:21.881656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad554dae-060c-4f52-85d2-c38016bfc9b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:21.882831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad554dae-060c-4f52-85d2-c38016bfc9b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:21.884269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad554dae-060c-4f52-85d2-c38016bfc9b7 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:21.886134 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 511/1010] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:21.891599 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635f2c5-f3eb-4151-af9e-8607611ff758 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:21.993596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635f2c5-f3eb-4151-af9e-8607611ff758 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:22.020492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:22.028880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f5023-37d1-484b-9371-73facda5f20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:22.031450 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f5023-37d1-484b-9371-73facda5f20f None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:22.031450 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f5023-37d1-484b-9371-73facda5f20f None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:22.031450 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f5023-37d1-484b-9371-73facda5f20f None admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:22.031450 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f5023-37d1-484b-9371-73facda5f20f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:22.032285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f5023-37d1-484b-9371-73facda5f20f None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:22.132741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 500/1011] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 22:28:21 2022] POST /identity/v3/projects => generated 311 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:28:22.191683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02e600-986e-40e0-84d7-00739c36e19d None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:22.192378 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02e600-986e-40e0-84d7-00739c36e19d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:22.192378 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02e600-986e-40e0-84d7-00739c36e19d None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:22.193864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 512/1012] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:22.199701 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4fa60-ae3f-4945-a6cf-ae28ad8eb39c None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:22.199994 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4fa60-ae3f-4945-a6cf-ae28ad8eb39c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:22.200276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4fa60-ae3f-4945-a6cf-ae28ad8eb39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:22.218880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-78a4fa60-ae3f-4945-a6cf-ae28ad8eb39c None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:22.220116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-78a4fa60-ae3f-4945-a6cf-ae28ad8eb39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:22.302017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 501/1013] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:22 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:22.326016 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02e600-986e-40e0-84d7-00739c36e19d None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:22.411306 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:22.433841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02e600-986e-40e0-84d7-00739c36e19d None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:22.468941 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0f219-417d-4332-bf58-9656c6df8ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:22.469740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0f219-417d-4332-bf58-9656c6df8ffd None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:22.470031 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0f219-417d-4332-bf58-9656c6df8ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:22.470309 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0f219-417d-4332-bf58-9656c6df8ffd None admin] PATH_INFO: `/v3/services/compute` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:22.471049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f0f219-417d-4332-bf58-9656c6df8ffd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:22.472419 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f0f219-417d-4332-bf58-9656c6df8ffd None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:22.484812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-e8f0f219-417d-4332-bf58-9656c6df8ffd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 22:28:22.484812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:22.484812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:28:22.484812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:28:22.484812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:28:22.484812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:28:22.484812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:28:22.484812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:22.484812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 22:28:22.484812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:22.484812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:28:22.484812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:22.484812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:22.484812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:22.484812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:22.484812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:22.484812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:22.485808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:22.485808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:22.485808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:22.485808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:22.485808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:22.485808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:22.485808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:28:22.485808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:28:22.485808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:28:22.485808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:28:22.485808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:22.485808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:22.485808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:22.485808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:22.485808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:22.485808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:22.486729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:22.486729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:22.486729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:22.486729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:22.486729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:22.486729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:22.486729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:22.486729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:22.486729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:22.486729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:22.486729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:28:22.486729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:22.486729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 22:28:22.486729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:22.486729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 513/1014] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:28:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:22.492239 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4fa60-ae3f-4945-a6cf-ae28ad8eb39c None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:22.595538 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4fa60-ae3f-4945-a6cf-ae28ad8eb39c None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:22.606620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:22.630793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3b8dd-249a-4528-87e8-614af8627bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:22.631480 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3b8dd-249a-4528-87e8-614af8627bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:22.631764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3b8dd-249a-4528-87e8-614af8627bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:22.632076 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3b8dd-249a-4528-87e8-614af8627bc4 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:22.632795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3b8dd-249a-4528-87e8-614af8627bc4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:22.633106 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3b8dd-249a-4528-87e8-614af8627bc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:22.634518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3b8dd-249a-4528-87e8-614af8627bc4 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:22.640260 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 502/1015] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 22:28:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:22.646351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0f219-417d-4332-bf58-9656c6df8ffd None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:22.750660 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0f219-417d-4332-bf58-9656c6df8ffd None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:22.783772 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a06a2-f05c-42cc-95be-6285d817e2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:22.784433 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a06a2-f05c-42cc-95be-6285d817e2cb None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:22.784746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a06a2-f05c-42cc-95be-6285d817e2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:22.785021 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a06a2-f05c-42cc-95be-6285d817e2cb None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:22.786310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a06a2-f05c-42cc-95be-6285d817e2cb None admin] RBAC: Adding query filter params (type=compute) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:22.786613 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a06a2-f05c-42cc-95be-6285d817e2cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:22.787970 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a06a2-f05c-42cc-95be-6285d817e2cb None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:22.793215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 514/1016] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 23 22:28:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:22.798609 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3b8dd-249a-4528-87e8-614af8627bc4 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:22.900655 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3b8dd-249a-4528-87e8-614af8627bc4 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:22.934804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908d2ad-51bc-4497-8877-85ae04de5989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:22.935470 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908d2ad-51bc-4497-8877-85ae04de5989 None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:22.935757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908d2ad-51bc-4497-8877-85ae04de5989 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:22.936032 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908d2ad-51bc-4497-8877-85ae04de5989 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:22.936863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908d2ad-51bc-4497-8877-85ae04de5989 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:22.938226 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908d2ad-51bc-4497-8877-85ae04de5989 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:23.033156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 503/1017] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 23 22:28:22 2022] POST /identity/v3/endpoints => generated 334 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:28:23.571110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e0a0d-cf19-43aa-9459-87fac6fcfc74 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:23.571626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e0a0d-cf19-43aa-9459-87fac6fcfc74 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:23.571626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e0a0d-cf19-43aa-9459-87fac6fcfc74 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:23.573112 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 515/1018] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:23.578011 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1badbab-a05e-41d6-be21-fc973e106758 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:23.578306 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1badbab-a05e-41d6-be21-fc973e106758 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:23.578587 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1badbab-a05e-41d6-be21-fc973e106758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:23.595094 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-e1badbab-a05e-41d6-be21-fc973e106758 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:23.596268 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e1badbab-a05e-41d6-be21-fc973e106758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:23.620659 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:23.670144 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 504/1019] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:23 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:23.693554 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e0a0d-cf19-43aa-9459-87fac6fcfc74 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:23.702884 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:23.798750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e0a0d-cf19-43aa-9459-87fac6fcfc74 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:23.833747 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c63368-b5c5-4155-bc09-3ccea2c7a8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:23.834356 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c63368-b5c5-4155-bc09-3ccea2c7a8a3 None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:23.834645 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c63368-b5c5-4155-bc09-3ccea2c7a8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:23.834924 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c63368-b5c5-4155-bc09-3ccea2c7a8a3 None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:23.835653 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c63368-b5c5-4155-bc09-3ccea2c7a8a3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:23.837045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c63368-b5c5-4155-bc09-3ccea2c7a8a3 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:23.932609 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 516/1020] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 22:28:23 2022] POST /identity/v3/services => generated 242 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:28:24.266003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d53a60-ea1b-495e-82c1-470558d5a6ee None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:24.266524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d53a60-ea1b-495e-82c1-470558d5a6ee None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:24.266602 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d53a60-ea1b-495e-82c1-470558d5a6ee None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:24.268654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 505/1021] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:24.274245 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cfb87-7b17-4402-a208-e238fb1d70c2 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:24.274539 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cfb87-7b17-4402-a208-e238fb1d70c2 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:24.274819 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cfb87-7b17-4402-a208-e238fb1d70c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:24.293894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-7f0cfb87-7b17-4402-a208-e238fb1d70c2 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:24.295083 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-7f0cfb87-7b17-4402-a208-e238fb1d70c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:24.355753 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:24.370265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 517/1022] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:24.395339 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d53a60-ea1b-495e-82c1-470558d5a6ee None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:24.499766 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d53a60-ea1b-495e-82c1-470558d5a6ee None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:24.533916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795bc7c-27c3-4b60-8f9c-a5021ac403c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:24.534636 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795bc7c-27c3-4b60-8f9c-a5021ac403c8 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:24.534926 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795bc7c-27c3-4b60-8f9c-a5021ac403c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:24.535221 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795bc7c-27c3-4b60-8f9c-a5021ac403c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:24.536797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a795bc7c-27c3-4b60-8f9c-a5021ac403c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:24.538234 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a795bc7c-27c3-4b60-8f9c-a5021ac403c8 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:24.540270 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 506/1023] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:24.545720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cfb87-7b17-4402-a208-e238fb1d70c2 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:24.645214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cfb87-7b17-4402-a208-e238fb1d70c2 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:24.679728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43212e17-05df-42da-a0ed-1a9a1ad927b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:24.680404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43212e17-05df-42da-a0ed-1a9a1ad927b2 None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:24.680725 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43212e17-05df-42da-a0ed-1a9a1ad927b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:24.681010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43212e17-05df-42da-a0ed-1a9a1ad927b2 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:24.681736 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43212e17-05df-42da-a0ed-1a9a1ad927b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:24.683201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43212e17-05df-42da-a0ed-1a9a1ad927b2 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:24.789200 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 518/1024] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 22:28:24 2022] POST /identity/v3/users => generated 285 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:28:25.184669 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e904e-65e9-4f2c-abe4-f306640bdb95 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:25.185029 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e904e-65e9-4f2c-abe4-f306640bdb95 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:25.185206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e904e-65e9-4f2c-abe4-f306640bdb95 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:25.186684 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 507/1025] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:25.192666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9c337-8413-4f73-90ee-a30eea1555bb None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:25.192970 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9c337-8413-4f73-90ee-a30eea1555bb None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:25.193252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9c337-8413-4f73-90ee-a30eea1555bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:25.209946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-04d9c337-8413-4f73-90ee-a30eea1555bb None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:25.211136 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-04d9c337-8413-4f73-90ee-a30eea1555bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:25.255655 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:25.283567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 519/1026] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:25.308858 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e904e-65e9-4f2c-abe4-f306640bdb95 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:25.336392 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:25.415824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e904e-65e9-4f2c-abe4-f306640bdb95 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:25.450303 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b4148-d1d0-4739-836e-812e3f5f3e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:25.451032 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b4148-d1d0-4739-836e-812e3f5f3e37 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:25.451314 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b4148-d1d0-4739-836e-812e3f5f3e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:25.451586 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b4148-d1d0-4739-836e-812e3f5f3e37 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:25.461824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b4148-d1d0-4739-836e-812e3f5f3e37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:28:25.469952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-cd3b4148-d1d0-4739-836e-812e3f5f3e37 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:28:25.469952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:25.469952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:25.469952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:25.469952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:25.469952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:25.469952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:25.469952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:25.469952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:25.469952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:25.469952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:25.469952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:25.469952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:28:25.469952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:28:25.469952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:28:25.469952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:28:25.470893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:28:25.470893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:28:25.470893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:28:25.470893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:28:25.470893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:25.470893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:25.470893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:25.470893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:25.470893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:25.470893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:25.470893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:25.470893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:25.470893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:25.470893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:25.470893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:25.470893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:25.477155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:25.477155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:25.477155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:25.477155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:25.477155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:28:25.477155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:28:25.477155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:28:25.477155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:28:25.477155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:28:25.477155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:28:25.477155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:28:25.477155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:25.477155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 508/1027] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 23 22:28:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:25.480069 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9c337-8413-4f73-90ee-a30eea1555bb None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:25.584434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9c337-8413-4f73-90ee-a30eea1555bb None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:25.619018 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c3868-8574-4598-9a17-afa4f760681e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:25.619754 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c3868-8574-4598-9a17-afa4f760681e None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:25.620044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c3868-8574-4598-9a17-afa4f760681e None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:25.620362 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c3868-8574-4598-9a17-afa4f760681e None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:25.621138 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c3868-8574-4598-9a17-afa4f760681e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:25.621452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c3868-8574-4598-9a17-afa4f760681e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:25.622815 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c3868-8574-4598-9a17-afa4f760681e None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:25.635484 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 520/1028] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 23 22:28:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:25.636936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b4148-d1d0-4739-836e-812e3f5f3e37 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:25.724067 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:25.740203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b4148-d1d0-4739-836e-812e3f5f3e37 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:25.773513 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b100b-43fb-4327-b5bd-784d72d92c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:25.774306 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b100b-43fb-4327-b5bd-784d72d92c34 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:25.774587 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b100b-43fb-4327-b5bd-784d72d92c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:25.774867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b100b-43fb-4327-b5bd-784d72d92c34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:25.784759 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b100b-43fb-4327-b5bd-784d72d92c34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:25.786767 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b100b-43fb-4327-b5bd-784d72d92c34 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:25.804966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-e58b100b-43fb-4327-b5bd-784d72d92c34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:25.804966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:25.804966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:25.804966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:25.804966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:25.804966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:25.804966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:25.804966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:25.804966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:25.804966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:25.804966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:25.804966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:25.804966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:28:25.804966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:28:25.804966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:28:25.805931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:28:25.805931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:25.805931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:25.805931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:25.805931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:25.805931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:25.805931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:25.805931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:25.805931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:25.805931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:25.805931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:25.805931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:25.805931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:25.805931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:25.805931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:25.805931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:25.805931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:25.811578 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:28:25.811578 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:28:25.811578 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:25.811578 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:25.811578 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 509/1029] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:25.813343 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c3868-8574-4598-9a17-afa4f760681e None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:25.857849 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:25.917363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c3868-8574-4598-9a17-afa4f760681e None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:25.951676 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e04eda-1ac0-434f-b38f-01f1112a41f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:25.952402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e04eda-1ac0-434f-b38f-01f1112a41f1 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:25.952714 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e04eda-1ac0-434f-b38f-01f1112a41f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:25.953151 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e04eda-1ac0-434f-b38f-01f1112a41f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:25.953912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e04eda-1ac0-434f-b38f-01f1112a41f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:25.954220 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e04eda-1ac0-434f-b38f-01f1112a41f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:25.955640 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e04eda-1ac0-434f-b38f-01f1112a41f1 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:25.972658 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 521/1030] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:25.975128 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b100b-43fb-4327-b5bd-784d72d92c34 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:26.000800 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4160b5c-d338-4900-b74c-2b0970937787 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:26.001153 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4160b5c-d338-4900-b74c-2b0970937787 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:26.001377 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4160b5c-d338-4900-b74c-2b0970937787 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:26.002731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 522/1031] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:28:26.008375 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2a442-7869-4431-985e-7efbcead4120 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:26.008694 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2a442-7869-4431-985e-7efbcead4120 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:26.008978 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2a442-7869-4431-985e-7efbcead4120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:26.025660 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-28d2a442-7869-4431-985e-7efbcead4120 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:26.026846 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-28d2a442-7869-4431-985e-7efbcead4120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:26.081593 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b100b-43fb-4327-b5bd-784d72d92c34 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:26.101754 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 523/1032] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:28:26.117059 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffb933-260a-4e05-9053-31d0dd1d4189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:26.117852 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffb933-260a-4e05-9053-31d0dd1d4189 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:26.118141 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffb933-260a-4e05-9053-31d0dd1d4189 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:26.118419 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffb933-260a-4e05-9053-31d0dd1d4189 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:26.121676 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:26.126000 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2a442-7869-4431-985e-7efbcead4120 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:26.126281 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ffb933-260a-4e05-9053-31d0dd1d4189 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:26.127725 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ffb933-260a-4e05-9053-31d0dd1d4189 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:26.143462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-70ffb933-260a-4e05-9053-31d0dd1d4189 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:28:26.143462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:26.143462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:26.143462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:26.143462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:26.143462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:26.143462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:26.143462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:26.143462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:26.143462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:26.143462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:26.143462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:26.143462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:28:26.143462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:28:26.143462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:28:26.143462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:28:26.144473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:26.144473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:26.144473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:28:26.144473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:28:26.144473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:28:26.144473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:28:26.144473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:26.144473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:26.144473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:26.144473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:26.144473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:26.144473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:26.144473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:26.144473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:26.144473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:26.144473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:26.144473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:26.144473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:26.145509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:26.145509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:26.145509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:28:26.145509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:28:26.145509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:28:26.145509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:28:26.145509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:28:26.145509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:28:26.145509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:28:26.145509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:26.145509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 510/1033] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 23 22:28:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:26.150341 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffb933-260a-4e05-9053-31d0dd1d4189 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:26.161700 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:26.237802 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2a442-7869-4431-985e-7efbcead4120 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:26.252538 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffb933-260a-4e05-9053-31d0dd1d4189 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:26.277784 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28edd85-3713-4f3a-a8f0-3875314f049f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:26.278584 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28edd85-3713-4f3a-a8f0-3875314f049f None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:26.278869 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28edd85-3713-4f3a-a8f0-3875314f049f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:26.279146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28edd85-3713-4f3a-a8f0-3875314f049f None admin] PATH_INFO: `/v3/services/image` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:26.279809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28edd85-3713-4f3a-a8f0-3875314f049f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:26.281203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28edd85-3713-4f3a-a8f0-3875314f049f None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:26.295061 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74babd9-d735-4982-bacd-ee83e348dbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:26.295729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74babd9-d735-4982-bacd-ee83e348dbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:26.296012 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74babd9-d735-4982-bacd-ee83e348dbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:26.296285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74babd9-d735-4982-bacd-ee83e348dbb8 None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:26.296382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-b28edd85-3713-4f3a-a8f0-3875314f049f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 22:28:26.296382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:26.296382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:28:26.296382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:28:26.296382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:28:26.296382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:28:26.296382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:28:26.296382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:26.296382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 22:28:26.296382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:26.296382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:28:26.296382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:26.296382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:26.296382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:26.296382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:26.296382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:26.296382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:26.297337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:26.297337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:26.297337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:26.297337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:26.297337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:26.297337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:26.297337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:28:26.297337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:28:26.297337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:28:26.297337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:28:26.297337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:26.297337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:26.297337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:26.297337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:26.297337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:26.297337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:26.301794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:26.301794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:26.301794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:26.301794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:26.301794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:26.301794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:26.301794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:26.301794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:26.301794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:26.301794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:26.301794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:28:26.301794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:26.301794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 22:28:26.301794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:26.302762 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74babd9-d735-4982-bacd-ee83e348dbb8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:26.302762 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74babd9-d735-4982-bacd-ee83e348dbb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:26.302906 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 524/1034] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Jun 23 22:28:26 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:26.302982 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74babd9-d735-4982-bacd-ee83e348dbb8 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:26.304089 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28edd85-3713-4f3a-a8f0-3875314f049f None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:26.319096 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 511/1035] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 23 22:28:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:26.324700 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74babd9-d735-4982-bacd-ee83e348dbb8 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:26.384561 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:26.409504 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:26.418411 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28edd85-3713-4f3a-a8f0-3875314f049f None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:26.431761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74babd9-d735-4982-bacd-ee83e348dbb8 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:26.459774 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79df84-1296-4f2e-bedf-f23bd808ea72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:26.460448 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79df84-1296-4f2e-bedf-f23bd808ea72 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:26.460766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79df84-1296-4f2e-bedf-f23bd808ea72 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:26.461041 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79df84-1296-4f2e-bedf-f23bd808ea72 None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:26.461745 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79df84-1296-4f2e-bedf-f23bd808ea72 None admin] RBAC: Adding query filter params (name=image) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:26.462051 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79df84-1296-4f2e-bedf-f23bd808ea72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:26.463449 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79df84-1296-4f2e-bedf-f23bd808ea72 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:26.468351 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e5d96-d279-42ae-b5a3-fa684b2a3238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:26.468491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 525/1036] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:26.469366 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e5d96-d279-42ae-b5a3-fa684b2a3238 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:26.469657 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e5d96-d279-42ae-b5a3-fa684b2a3238 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:26.469941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e5d96-d279-42ae-b5a3-fa684b2a3238 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:26.474517 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79df84-1296-4f2e-bedf-f23bd808ea72 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:26.548336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e5d96-d279-42ae-b5a3-fa684b2a3238 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:26.549776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e5d96-d279-42ae-b5a3-fa684b2a3238 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:26.576961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79df84-1296-4f2e-bedf-f23bd808ea72 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:26.608268 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:26.612280 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0588422-ccb4-406c-9b1f-a6e619b3b8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:26.612976 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0588422-ccb4-406c-9b1f-a6e619b3b8bb None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:26.613873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0588422-ccb4-406c-9b1f-a6e619b3b8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:26.614154 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0588422-ccb4-406c-9b1f-a6e619b3b8bb None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:26.614854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0588422-ccb4-406c-9b1f-a6e619b3b8bb None admin] RBAC: Adding query filter params (type=image) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:26.615157 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0588422-ccb4-406c-9b1f-a6e619b3b8bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:26.616605 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0588422-ccb4-406c-9b1f-a6e619b3b8bb None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:26.621607 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 526/1037] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:26.627241 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0588422-ccb4-406c-9b1f-a6e619b3b8bb None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:26.651002 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:26.658667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 512/1038] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:28:26 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 22:28:26.665390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e5d96-d279-42ae-b5a3-fa684b2a3238 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:26.731599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0588422-ccb4-406c-9b1f-a6e619b3b8bb None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:26.766503 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e46221-dbac-4282-9a31-a3abe425ee77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:26.767167 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e46221-dbac-4282-9a31-a3abe425ee77 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:26.767450 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e46221-dbac-4282-9a31-a3abe425ee77 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:26.767726 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e46221-dbac-4282-9a31-a3abe425ee77 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:26.768502 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e46221-dbac-4282-9a31-a3abe425ee77 None admin] RBAC: Adding query filter params (interface=public, service_id=98e96cde3a4740c8ae426e059047dd0e, region_id=RegionOne) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:26.768863 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e46221-dbac-4282-9a31-a3abe425ee77 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:26.770771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e46221-dbac-4282-9a31-a3abe425ee77 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:26.772099 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e5d96-d279-42ae-b5a3-fa684b2a3238 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:26.776459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 527/1039] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 23 22:28:26 2022] GET /identity/v3/endpoints?service_id=98e96cde3a4740c8ae426e059047dd0e&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:26.781838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e46221-dbac-4282-9a31-a3abe425ee77 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:26.810657 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f83ee-3a93-4925-8e43-f06da4ca0cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:26.811380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f83ee-3a93-4925-8e43-f06da4ca0cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:26.811667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f83ee-3a93-4925-8e43-f06da4ca0cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:26.811942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f83ee-3a93-4925-8e43-f06da4ca0cc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:26.823210 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f83ee-3a93-4925-8e43-f06da4ca0cc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:26.824677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f83ee-3a93-4925-8e43-f06da4ca0cc3 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:26.843185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-e48f83ee-3a93-4925-8e43-f06da4ca0cc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:26.843185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:26.843185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:26.843185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:26.843185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:26.843185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:26.843185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:26.843185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:26.843185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:26.843185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:26.843185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:26.843185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:26.843185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:28:26.843185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:28:26.843185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:28:26.844145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:28:26.844145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:26.844145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:26.844145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:26.844145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:26.844145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:26.844145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:26.844145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:26.844145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:26.844145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:26.844145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:26.844145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:26.844145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:26.844145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:26.844145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:26.844145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:26.844145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:26.845224 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:28:26.845224 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:28:26.845224 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:26.845224 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:26.845224 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 513/1040] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:26.851013 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f83ee-3a93-4925-8e43-f06da4ca0cc3 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:26.860825 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:26.876842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:26.892273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e46221-dbac-4282-9a31-a3abe425ee77 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:26.929864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b26e6-bd34-494f-8609-728b73d4bc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:26.930521 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b26e6-bd34-494f-8609-728b73d4bc5e None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:26.930804 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b26e6-bd34-494f-8609-728b73d4bc5e None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:26.931077 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b26e6-bd34-494f-8609-728b73d4bc5e None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:26.931754 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b26e6-bd34-494f-8609-728b73d4bc5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:26.933167 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b26e6-bd34-494f-8609-728b73d4bc5e None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:26.939595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 528/1041] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 23 22:28:26 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:28:26.942051 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a68b74-823c-488e-8f13-100b4fd96f6b None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:26.942346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a68b74-823c-488e-8f13-100b4fd96f6b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:26.942633 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a68b74-823c-488e-8f13-100b4fd96f6b None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:26.943977 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 529/1042] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:26.949910 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b933e-b432-4539-93bb-fdf5a20f1093 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:26.950450 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b933e-b432-4539-93bb-fdf5a20f1093 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:26.950737 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b933e-b432-4539-93bb-fdf5a20f1093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:26.963352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f83ee-3a93-4925-8e43-f06da4ca0cc3 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:26.969412 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-be0b933e-b432-4539-93bb-fdf5a20f1093 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:26.970660 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-be0b933e-b432-4539-93bb-fdf5a20f1093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:27.002450 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c35ef1-621a-4672-8988-0564c6883e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:27.003137 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c35ef1-621a-4672-8988-0564c6883e7b None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:27.003422 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c35ef1-621a-4672-8988-0564c6883e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:27.004227 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c35ef1-621a-4672-8988-0564c6883e7b None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:27.005689 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c35ef1-621a-4672-8988-0564c6883e7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:27.006288 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c35ef1-621a-4672-8988-0564c6883e7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:27.007881 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c35ef1-621a-4672-8988-0564c6883e7b None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:27.021916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 514/1043] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:27.027722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c35ef1-621a-4672-8988-0564c6883e7b None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:27.052721 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 530/1044] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:27.077698 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b933e-b432-4539-93bb-fdf5a20f1093 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:27.111199 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:27.151454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c35ef1-621a-4672-8988-0564c6883e7b None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:27.171525 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:27.188561 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6e5e1-97a7-4544-86a9-a76a19edb79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:27.189364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6e5e1-97a7-4544-86a9-a76a19edb79b None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:27.189653 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6e5e1-97a7-4544-86a9-a76a19edb79b None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:27.189934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6e5e1-97a7-4544-86a9-a76a19edb79b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:27.191780 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b933e-b432-4539-93bb-fdf5a20f1093 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:27.197403 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6e5e1-97a7-4544-86a9-a76a19edb79b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:27.198864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6e5e1-97a7-4544-86a9-a76a19edb79b None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:27.215062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-f8d6e5e1-97a7-4544-86a9-a76a19edb79b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:28:27.215062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:27.215062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:27.215062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:27.215062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:27.215062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:27.215062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:27.215062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:27.215062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:27.215062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:27.215062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:27.215062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:27.215062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:28:27.215062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:28:27.215062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:28:27.216009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:28:27.216009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:27.216009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:27.216009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:27.216009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:27.216009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:27.216009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:27.216009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:27.216009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:27.216009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:27.216009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:27.216009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:27.216009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:27.216009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:27.216009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:27.216009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:27.216009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:27.216948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:28:27.216948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:28:27.216948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:28:27.216948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:28:27.216948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:28:27.216948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:28:27.216948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:28:27.216948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:27.216948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 515/1045] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:28:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:27.221910 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6e5e1-97a7-4544-86a9-a76a19edb79b None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:27.234057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0dd39-e545-4b1c-896f-b314e17eac5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:27.235173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0dd39-e545-4b1c-896f-b314e17eac5c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:27.235459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0dd39-e545-4b1c-896f-b314e17eac5c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:27.235736 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0dd39-e545-4b1c-896f-b314e17eac5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:27.247187 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0dd39-e545-4b1c-896f-b314e17eac5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:28:27.255495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-34d0dd39-e545-4b1c-896f-b314e17eac5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:28:27.255495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:27.255495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:27.255495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:27.255495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:27.255495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:27.255495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:27.255495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:27.255495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:27.255495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:27.255495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:27.255495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:27.255495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:28:27.255495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:28:27.255495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:28:27.255495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:28:27.256441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:28:27.256441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:28:27.256441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:28:27.256441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:28:27.256441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:27.256441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:27.256441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:27.256441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:27.256441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:27.256441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:27.256441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:27.256441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:27.256441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:27.256441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:27.256441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:27.256441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:27.262519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:27.262519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:27.262519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:27.262519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:27.262519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:28:27.262519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:28:27.262519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:28:27.262519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:28:27.262519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:28:27.262519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:28:27.262519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:28:27.262519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:27.262519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 531/1046] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:28:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:27.263317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0dd39-e545-4b1c-896f-b314e17eac5c None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:27.324639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6e5e1-97a7-4544-86a9-a76a19edb79b None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:27.351051 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:27.360376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181465a-25fa-40b9-94de-e013b11e1903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:27.361054 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181465a-25fa-40b9-94de-e013b11e1903 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:27.361922 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181465a-25fa-40b9-94de-e013b11e1903 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:27.361922 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181465a-25fa-40b9-94de-e013b11e1903 None admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:27.362573 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181465a-25fa-40b9-94de-e013b11e1903 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:27.362884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181465a-25fa-40b9-94de-e013b11e1903 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:27.365170 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181465a-25fa-40b9-94de-e013b11e1903 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:27.381779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 516/1047] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:28:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:27.382845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181465a-25fa-40b9-94de-e013b11e1903 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:27.387628 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0dd39-e545-4b1c-896f-b314e17eac5c None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:27.426290 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a75fbb-080c-46bb-8bca-85131caa0f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:27.426970 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a75fbb-080c-46bb-8bca-85131caa0f4a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:27.427257 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a75fbb-080c-46bb-8bca-85131caa0f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:27.427546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a75fbb-080c-46bb-8bca-85131caa0f4a None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:27.428304 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a75fbb-080c-46bb-8bca-85131caa0f4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:27.428640 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a75fbb-080c-46bb-8bca-85131caa0f4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:27.430021 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a75fbb-080c-46bb-8bca-85131caa0f4a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:27.438298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 532/1048] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:28:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:27.444490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a75fbb-080c-46bb-8bca-85131caa0f4a None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:27.447154 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:27.495294 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181465a-25fa-40b9-94de-e013b11e1903 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:27.531386 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70ea48-77dd-41d3-a48c-a6c732571123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:27.532039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70ea48-77dd-41d3-a48c-a6c732571123 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:27.532328 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70ea48-77dd-41d3-a48c-a6c732571123 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:27.532635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70ea48-77dd-41d3-a48c-a6c732571123 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:27.533497 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70ea48-77dd-41d3-a48c-a6c732571123 None admin] RBAC: Adding query filter params (role.id=b1cc8597c7b144ed9b6bed9fc40f3c97, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=159cc8befc3a4d57bbd313832084db62) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:27.533811 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70ea48-77dd-41d3-a48c-a6c732571123 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:27.535318 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70ea48-77dd-41d3-a48c-a6c732571123 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:27.541068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 517/1049] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:28:27 2022] GET /identity/v3/role_assignments?user.id=159cc8befc3a4d57bbd313832084db62&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=b1cc8597c7b144ed9b6bed9fc40f3c97 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:27.564072 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a75fbb-080c-46bb-8bca-85131caa0f4a None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:27.602253 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692f621-f480-4923-9be4-f9c68b4c0130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:27.603067 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692f621-f480-4923-9be4-f9c68b4c0130 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:27.603351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692f621-f480-4923-9be4-f9c68b4c0130 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:27.603628 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692f621-f480-4923-9be4-f9c68b4c0130 None admin] PATH_INFO: `/v3/users/5c2a357ac26346219d2b2211edc747db` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:27.619446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f692f621-f480-4923-9be4-f9c68b4c0130 None admin] RBAC: Authorizing `identity:get_user(user_id=5c2a357ac26346219d2b2211edc747db)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:27.620951 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f692f621-f480-4923-9be4-f9c68b4c0130 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:27.622904 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 533/1050] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:28:27 2022] GET /identity/v3/users/5c2a357ac26346219d2b2211edc747db => generated 285 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:27.628742 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70ea48-77dd-41d3-a48c-a6c732571123 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:27.682447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:27.736445 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70ea48-77dd-41d3-a48c-a6c732571123 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:27.773950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d024736-a94f-4e2d-9ddb-d73b2e70e675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:27.774611 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d024736-a94f-4e2d-9ddb-d73b2e70e675 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:27.774897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d024736-a94f-4e2d-9ddb-d73b2e70e675 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:27.775177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d024736-a94f-4e2d-9ddb-d73b2e70e675 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:27.845609 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d024736-a94f-4e2d-9ddb-d73b2e70e675 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:27.847089 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d024736-a94f-4e2d-9ddb-d73b2e70e675 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:27.906103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:27.969176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 518/1051] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:28:27 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:28:27.976699 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692f621-f480-4923-9be4-f9c68b4c0130 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:28.039406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:28.084384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692f621-f480-4923-9be4-f9c68b4c0130 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:28.120800 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48dc80-c4c8-42b8-ba1f-abdf9241ce87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:28.121669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48dc80-c4c8-42b8-ba1f-abdf9241ce87 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:28.121952 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48dc80-c4c8-42b8-ba1f-abdf9241ce87 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:28.122534 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48dc80-c4c8-42b8-ba1f-abdf9241ce87 None admin] PATH_INFO: `/v3/projects/53a6be8ef92244f9bcfe40cf07df3f4e` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:28.123747 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48dc80-c4c8-42b8-ba1f-abdf9241ce87 None admin] RBAC: Authorizing `identity:get_project(project_id=53a6be8ef92244f9bcfe40cf07df3f4e)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:28.125212 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48dc80-c4c8-42b8-ba1f-abdf9241ce87 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:28.132026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 534/1052] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:28:27 2022] GET /identity/v3/projects/53a6be8ef92244f9bcfe40cf07df3f4e => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:28.133461 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d024736-a94f-4e2d-9ddb-d73b2e70e675 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:28.236109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d024736-a94f-4e2d-9ddb-d73b2e70e675 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:28.273829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965f607-3882-4836-80d4-84f6693e66ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:28.274685 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965f607-3882-4836-80d4-84f6693e66ab None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:28.275008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965f607-3882-4836-80d4-84f6693e66ab None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:28.275291 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965f607-3882-4836-80d4-84f6693e66ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:28.276142 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965f607-3882-4836-80d4-84f6693e66ab None admin] RBAC: Adding query filter params (role.id=6ff7708b59984b9287cc81d46a57b4b4, scope.project.id=53a6be8ef92244f9bcfe40cf07df3f4e, user.id=5c2a357ac26346219d2b2211edc747db) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:28.276463 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965f607-3882-4836-80d4-84f6693e66ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:28.277951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965f607-3882-4836-80d4-84f6693e66ab None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:28.286782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 519/1053] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:28:28 2022] GET /identity/v3/role_assignments?user.id=5c2a357ac26346219d2b2211edc747db&scope.project.id=53a6be8ef92244f9bcfe40cf07df3f4e&role.id=6ff7708b59984b9287cc81d46a57b4b4 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:29.077413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ea022-47b6-4ad2-b5c4-a1bda5c1f925 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:29.077943 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ea022-47b6-4ad2-b5c4-a1bda5c1f925 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:29.077943 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ea022-47b6-4ad2-b5c4-a1bda5c1f925 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:29.079453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 535/1054] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:29.085053 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a464be6-5599-43b3-a511-320489e33dbc None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:29.085455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a464be6-5599-43b3-a511-320489e33dbc None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:29.085625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a464be6-5599-43b3-a511-320489e33dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:29.102646 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-2a464be6-5599-43b3-a511-320489e33dbc None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:29.103824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-2a464be6-5599-43b3-a511-320489e33dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:29.113261 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:29.172735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 520/1055] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:29.194079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ea022-47b6-4ad2-b5c4-a1bda5c1f925 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:29.301789 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ea022-47b6-4ad2-b5c4-a1bda5c1f925 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:29.334359 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:29.337347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141ec8c-e2c9-4ee5-a33a-4cbf10978af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:29.338138 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141ec8c-e2c9-4ee5-a33a-4cbf10978af7 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:29.338420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141ec8c-e2c9-4ee5-a33a-4cbf10978af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:29.338692 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141ec8c-e2c9-4ee5-a33a-4cbf10978af7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:29.339337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141ec8c-e2c9-4ee5-a33a-4cbf10978af7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:29.340739 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141ec8c-e2c9-4ee5-a33a-4cbf10978af7 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:29.352590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-9141ec8c-e2c9-4ee5-a33a-4cbf10978af7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 22:28:29.352590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:29.352590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:28:29.352590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:28:29.352590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:28:29.352590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:28:29.352590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:28:29.352590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:29.352590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 22:28:29.352590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:29.352590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:28:29.352590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:29.352590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:29.352590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:29.352590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:29.352590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:29.352590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:29.355818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:29.355818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:29.355818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:29.355818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:29.355818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:29.355818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:29.355818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:28:29.355818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:28:29.355818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:28:29.355818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:28:29.355818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:29.355818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:29.355818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:29.355818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:29.355818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:29.355818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:29.356982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:29.356982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:29.356982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:29.356982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:29.356982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:29.356982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:29.356982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:29.356982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:29.356982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:29.356982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:29.356982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:28:29.356982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:28:29.356982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 22:28:29.356982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:29.356982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 536/1056] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Jun 23 22:28:29 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:29.359774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a464be6-5599-43b3-a511-320489e33dbc None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:29.462638 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a464be6-5599-43b3-a511-320489e33dbc None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:29.496157 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12899f-4f35-4b5c-a5e5-f17916230835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:29.496850 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12899f-4f35-4b5c-a5e5-f17916230835 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:29.497135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12899f-4f35-4b5c-a5e5-f17916230835 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:29.497411 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12899f-4f35-4b5c-a5e5-f17916230835 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:29.498108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12899f-4f35-4b5c-a5e5-f17916230835 None admin] RBAC: Adding query filter params (name=image) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:29.498413 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12899f-4f35-4b5c-a5e5-f17916230835 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:29.499769 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12899f-4f35-4b5c-a5e5-f17916230835 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:29.510048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 521/1057] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:29.510984 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141ec8c-e2c9-4ee5-a33a-4cbf10978af7 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:29.611886 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141ec8c-e2c9-4ee5-a33a-4cbf10978af7 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:29.645563 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac0dc2-90b4-41df-9cb9-66ca83f15fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:29.646216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac0dc2-90b4-41df-9cb9-66ca83f15fad None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:29.646497 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac0dc2-90b4-41df-9cb9-66ca83f15fad None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:29.646770 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac0dc2-90b4-41df-9cb9-66ca83f15fad None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:29.647472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac0dc2-90b4-41df-9cb9-66ca83f15fad None admin] RBAC: Adding query filter params (type=image) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:29.647776 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac0dc2-90b4-41df-9cb9-66ca83f15fad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:29.649157 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac0dc2-90b4-41df-9cb9-66ca83f15fad None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:29.654103 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 537/1058] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:29 2022] GET /identity/v3/services?type=image => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:29.659666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12899f-4f35-4b5c-a5e5-f17916230835 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:29.670841 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:29.759500 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12899f-4f35-4b5c-a5e5-f17916230835 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:29.784161 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4262e3-e6ec-44f6-a656-aa249f3bb71e None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:29.784448 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4262e3-e6ec-44f6-a656-aa249f3bb71e None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:29.785052 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4262e3-e6ec-44f6-a656-aa249f3bb71e None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:29.786434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 538/1059] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:28:29.791562 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f37d8-4ed2-4767-91cb-be66e1499c7d None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:29.791867 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f37d8-4ed2-4767-91cb-be66e1499c7d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:29.792135 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f37d8-4ed2-4767-91cb-be66e1499c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:29.795881 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f66c7-b9b9-4552-a751-7d565c2d6577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:29.796542 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f66c7-b9b9-4552-a751-7d565c2d6577 None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:29.796859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f66c7-b9b9-4552-a751-7d565c2d6577 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:29.797306 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f66c7-b9b9-4552-a751-7d565c2d6577 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:29.798870 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f66c7-b9b9-4552-a751-7d565c2d6577 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:29.800256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f66c7-b9b9-4552-a751-7d565c2d6577 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:29.809678 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-e31f37d8-4ed2-4767-91cb-be66e1499c7d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:29.810965 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e31f37d8-4ed2-4767-91cb-be66e1499c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:29.860277 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:29.872411 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 522/1060] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 23 22:28:29 2022] POST /identity/v3/endpoints => generated 327 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:28:29.892610 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 539/1061] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:28:29.915697 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f66c7-b9b9-4552-a751-7d565c2d6577 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:29.985223 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:30.028677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f66c7-b9b9-4552-a751-7d565c2d6577 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:30.066466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4a189-d29f-4ebd-967a-1dc5f2ce4fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:30.067353 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4a189-d29f-4ebd-967a-1dc5f2ce4fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:30.067642 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4a189-d29f-4ebd-967a-1dc5f2ce4fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:30.067966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4a189-d29f-4ebd-967a-1dc5f2ce4fc9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:30.079129 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4a189-d29f-4ebd-967a-1dc5f2ce4fc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:28:30.087380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-04c4a189-d29f-4ebd-967a-1dc5f2ce4fc9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:28:30.087380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:30.087380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:30.087380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:30.087380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:30.087380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:30.087380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:30.087380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:30.087380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:30.087380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:30.087380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:30.087380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:30.087380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:28:30.087380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:28:30.087380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:28:30.087380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:28:30.088833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:28:30.088833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:28:30.088833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:28:30.088833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:28:30.088833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:30.088833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:30.088833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:30.088833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:30.088833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:30.088833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:30.088833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:30.088833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:30.088833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:30.088833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:30.088833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:30.088833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:30.089822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:30.089822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:30.089822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:30.089822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:30.089822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:28:30.089822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:28:30.089822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:28:30.089822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:28:30.089822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:28:30.089822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:28:30.089822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:28:30.089822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:30.089822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 523/1062] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 23 22:28:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:30.097218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f37d8-4ed2-4767-91cb-be66e1499c7d None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:30.201518 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f37d8-4ed2-4767-91cb-be66e1499c7d None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:30.235945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68873e2c-fe7e-4520-b0dc-4d6aa24408f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:30.236708 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68873e2c-fe7e-4520-b0dc-4d6aa24408f1 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:30.237000 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68873e2c-fe7e-4520-b0dc-4d6aa24408f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:30.237280 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68873e2c-fe7e-4520-b0dc-4d6aa24408f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:30.238286 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68873e2c-fe7e-4520-b0dc-4d6aa24408f1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:30.238602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68873e2c-fe7e-4520-b0dc-4d6aa24408f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:30.239982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68873e2c-fe7e-4520-b0dc-4d6aa24408f1 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:30.248458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 540/1063] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 23 22:28:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:30.254850 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4a189-d29f-4ebd-967a-1dc5f2ce4fc9 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:30.365873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5c027-fa64-4164-9ce1-0f726c3dae73 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:30.366161 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5c027-fa64-4164-9ce1-0f726c3dae73 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:30.366440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5c027-fa64-4164-9ce1-0f726c3dae73 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:30.367966 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 541/1064] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:30.368568 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4a189-d29f-4ebd-967a-1dc5f2ce4fc9 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:30.373000 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d687f9-b051-477c-b6f7-5f4b69cc60ad None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:30.373291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d687f9-b051-477c-b6f7-5f4b69cc60ad None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:30.373571 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d687f9-b051-477c-b6f7-5f4b69cc60ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:30.391256 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-d9d687f9-b051-477c-b6f7-5f4b69cc60ad None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:30.392227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d9d687f9-b051-477c-b6f7-5f4b69cc60ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:30.396929 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:30.408735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bde284-7a47-46f2-a895-8b2ec4033305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:30.409443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bde284-7a47-46f2-a895-8b2ec4033305 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:30.409729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bde284-7a47-46f2-a895-8b2ec4033305 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:30.410594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bde284-7a47-46f2-a895-8b2ec4033305 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:30.421489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bde284-7a47-46f2-a895-8b2ec4033305 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:30.422944 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bde284-7a47-46f2-a895-8b2ec4033305 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:30.439451 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:30.442235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-79bde284-7a47-46f2-a895-8b2ec4033305 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:30.442235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:30.442235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:30.442235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:30.442235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:30.442235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:30.442235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:30.442235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:30.442235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:30.442235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:30.442235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:30.442235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:30.442235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:28:30.442235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:28:30.442235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:28:30.443127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:28:30.443127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:30.443127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:30.443127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:30.443127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:30.443127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:30.443127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:30.443127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:30.443127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:30.443127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:30.443127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:30.443127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:30.443127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:30.443127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:30.443127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:30.443127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:30.443127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:30.444033 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:28:30.444033 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:28:30.444033 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:30.444033 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:30.444033 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 524/1065] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:30.449083 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bde284-7a47-46f2-a895-8b2ec4033305 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:30.470703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 542/1066] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:28:30.493949 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d687f9-b051-477c-b6f7-5f4b69cc60ad None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:30.558777 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bde284-7a47-46f2-a895-8b2ec4033305 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:30.596052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514f08b-e1f5-4bfb-b3c6-ffdf0cc91d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:30.596749 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514f08b-e1f5-4bfb-b3c6-ffdf0cc91d50 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:30.597037 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514f08b-e1f5-4bfb-b3c6-ffdf0cc91d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:30.598262 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514f08b-e1f5-4bfb-b3c6-ffdf0cc91d50 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:30.598579 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514f08b-e1f5-4bfb-b3c6-ffdf0cc91d50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:30.598889 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514f08b-e1f5-4bfb-b3c6-ffdf0cc91d50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:30.600271 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514f08b-e1f5-4bfb-b3c6-ffdf0cc91d50 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:30.604247 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d687f9-b051-477c-b6f7-5f4b69cc60ad None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:30.612479 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 525/1067] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:30.618148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514f08b-e1f5-4bfb-b3c6-ffdf0cc91d50 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:30.639825 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88dded0-275e-4550-bad5-9c21dd782a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:30.640658 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88dded0-275e-4550-bad5-9c21dd782a7e None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:30.640950 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88dded0-275e-4550-bad5-9c21dd782a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:30.641657 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88dded0-275e-4550-bad5-9c21dd782a7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:30.652917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88dded0-275e-4550-bad5-9c21dd782a7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:28:30.655687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:30.661654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-c88dded0-275e-4550-bad5-9c21dd782a7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:28:30.661654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:30.661654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:30.661654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:30.661654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:30.661654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:30.661654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:30.661654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:30.661654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:30.661654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:30.661654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:30.661654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:30.661654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:28:30.661654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:28:30.661654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:28:30.661654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:28:30.662691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:28:30.662691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:28:30.662691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:28:30.662691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:28:30.662691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:30.662691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:30.662691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:30.662691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:30.662691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:30.662691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:30.662691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:30.662691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:30.662691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:30.662691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:30.662691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:30.662691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:30.666980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:30.666980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:30.666980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:30.666980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:30.666980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:28:30.666980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:28:30.666980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:28:30.666980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:28:30.666980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:28:30.666980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:28:30.666980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:28:30.666980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:30.666980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 543/1068] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:28:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:30.669881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88dded0-275e-4550-bad5-9c21dd782a7e None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:30.706239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:30.729980 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514f08b-e1f5-4bfb-b3c6-ffdf0cc91d50 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:30.768314 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee135be5-67be-46ec-8ce9-4377b6c4dbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:30.769143 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee135be5-67be-46ec-8ce9-4377b6c4dbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:30.769989 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee135be5-67be-46ec-8ce9-4377b6c4dbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:30.770310 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee135be5-67be-46ec-8ce9-4377b6c4dbf0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:30.775185 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88dded0-275e-4550-bad5-9c21dd782a7e None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:30.777496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee135be5-67be-46ec-8ce9-4377b6c4dbf0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:30.779112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee135be5-67be-46ec-8ce9-4377b6c4dbf0 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:30.794111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-ee135be5-67be-46ec-8ce9-4377b6c4dbf0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:28:30.794111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:30.794111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:30.794111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:30.794111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:30.794111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:30.794111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:30.794111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:30.794111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:30.794111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:30.794111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:30.794111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:30.794111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:28:30.794111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:28:30.794111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:28:30.794111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:28:30.795103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:30.795103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:30.795103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:28:30.795103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:28:30.795103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:28:30.795103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:28:30.795103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:30.795103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:30.795103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:30.795103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:30.795103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:30.795103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:30.795103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:30.795103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:30.795103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:30.795103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:30.795103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:30.795103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:30.796060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:30.796060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:30.796060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:28:30.796060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:28:30.796060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:28:30.796060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:28:30.796060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:28:30.796060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:28:30.796060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:28:30.796060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:30.796060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 526/1069] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 23 22:28:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:30.801443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee135be5-67be-46ec-8ce9-4377b6c4dbf0 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:30.815780 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc1fe3-5109-4335-a37e-51e47c6fe6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:30.816489 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc1fe3-5109-4335-a37e-51e47c6fe6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:30.816812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc1fe3-5109-4335-a37e-51e47c6fe6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:30.817092 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc1fe3-5109-4335-a37e-51e47c6fe6d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:30.817841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc1fe3-5109-4335-a37e-51e47c6fe6d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:30.818150 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc1fe3-5109-4335-a37e-51e47c6fe6d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:30.819516 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc1fe3-5109-4335-a37e-51e47c6fe6d4 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:30.828309 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 544/1070] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:28:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:30.833846 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc1fe3-5109-4335-a37e-51e47c6fe6d4 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:30.912601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:30.918789 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee135be5-67be-46ec-8ce9-4377b6c4dbf0 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:30.930821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:30.936753 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc1fe3-5109-4335-a37e-51e47c6fe6d4 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:30.956405 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229b607-bb77-49ca-b7da-7b305471b63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:30.957345 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229b607-bb77-49ca-b7da-7b305471b63a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:30.957631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229b607-bb77-49ca-b7da-7b305471b63a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:30.957910 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229b607-bb77-49ca-b7da-7b305471b63a None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:30.958746 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229b607-bb77-49ca-b7da-7b305471b63a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:30.959057 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229b607-bb77-49ca-b7da-7b305471b63a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:30.960500 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229b607-bb77-49ca-b7da-7b305471b63a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:30.973526 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc00f84-702d-46d1-927a-022e28154e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:30.974406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc00f84-702d-46d1-927a-022e28154e2f None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:30.974692 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc00f84-702d-46d1-927a-022e28154e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:30.974967 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc00f84-702d-46d1-927a-022e28154e2f None admin] PATH_INFO: `/v3/users/5c2a357ac26346219d2b2211edc747db` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:30.976183 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc00f84-702d-46d1-927a-022e28154e2f None admin] RBAC: Authorizing `identity:get_user(user_id=5c2a357ac26346219d2b2211edc747db)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:30.977634 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc00f84-702d-46d1-927a-022e28154e2f None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:30.979417 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 545/1071] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:28:30 2022] GET /identity/v3/users/5c2a357ac26346219d2b2211edc747db => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:30.980262 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 527/1072] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 23 22:28:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:30.985413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc00f84-702d-46d1-927a-022e28154e2f None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:30.985582 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229b607-bb77-49ca-b7da-7b305471b63a None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:31.087821 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229b607-bb77-49ca-b7da-7b305471b63a None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:31.102237 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc00f84-702d-46d1-927a-022e28154e2f None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:31.122293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aaef04-9dd6-46b5-a627-0311eaa852b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:31.123092 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aaef04-9dd6-46b5-a627-0311eaa852b4 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:31.123373 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aaef04-9dd6-46b5-a627-0311eaa852b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:31.123653 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aaef04-9dd6-46b5-a627-0311eaa852b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:31.133508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aaef04-9dd6-46b5-a627-0311eaa852b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:31.134934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aaef04-9dd6-46b5-a627-0311eaa852b4 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:31.138648 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f06eb7a-0782-4848-b0a6-b723582d012c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:31.139717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06eb7a-0782-4848-b0a6-b723582d012c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:31.140187 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06eb7a-0782-4848-b0a6-b723582d012c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:31.140675 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06eb7a-0782-4848-b0a6-b723582d012c None admin] PATH_INFO: `/v3/projects/53a6be8ef92244f9bcfe40cf07df3f4e` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:31.142184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f06eb7a-0782-4848-b0a6-b723582d012c None admin] RBAC: Authorizing `identity:get_project(project_id=53a6be8ef92244f9bcfe40cf07df3f4e)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:31.143928 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f06eb7a-0782-4848-b0a6-b723582d012c None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:31.151351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 546/1073] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:28:30 2022] GET /identity/v3/projects/53a6be8ef92244f9bcfe40cf07df3f4e => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:31.152336 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f06eb7a-0782-4848-b0a6-b723582d012c None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:31.152511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-90aaef04-9dd6-46b5-a627-0311eaa852b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:31.152511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:31.152511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:31.152511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:31.152511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:31.152511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:31.152511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:31.152511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:31.152511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:31.152511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:31.152511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:31.152511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:31.152511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:28:31.152511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:28:31.152511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:28:31.153402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:28:31.153402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:31.153402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:31.153402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:31.153402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:31.153402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:31.153402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:31.153402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:31.153402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:31.153402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:31.153402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:31.153402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:31.153402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:31.153402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:31.153402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:31.153402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:31.153402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:31.154322 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:28:31.154322 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:28:31.154322 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:31.154322 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:31.154322 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 528/1074] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:31.159060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aaef04-9dd6-46b5-a627-0311eaa852b4 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:31.168759 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:31.190203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:31.260204 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f06eb7a-0782-4848-b0a6-b723582d012c None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:31.263347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aaef04-9dd6-46b5-a627-0311eaa852b4 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:31.301934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ed41e-7872-4f03-809e-c2d4a00b6c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:31.302087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea6baf-d0db-4e1c-8a1d-f4c9d6529b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:31.302607 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea6baf-d0db-4e1c-8a1d-f4c9d6529b16 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:31.302825 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ed41e-7872-4f03-809e-c2d4a00b6c3d None admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:31.302923 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea6baf-d0db-4e1c-8a1d-f4c9d6529b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:31.303109 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ed41e-7872-4f03-809e-c2d4a00b6c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:31.303204 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea6baf-d0db-4e1c-8a1d-f4c9d6529b16 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:31.303382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ed41e-7872-4f03-809e-c2d4a00b6c3d None admin] PATH_INFO: `/v3/projects/53a6be8ef92244f9bcfe40cf07df3f4e/users/5c2a357ac26346219d2b2211edc747db/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:31.303856 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea6baf-d0db-4e1c-8a1d-f4c9d6529b16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:31.304162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea6baf-d0db-4e1c-8a1d-f4c9d6529b16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:31.305373 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ed41e-7872-4f03-809e-c2d4a00b6c3d None admin] RBAC: Authorizing `identity:create_grant(project_id=53a6be8ef92244f9bcfe40cf07df3f4e, user_id=5c2a357ac26346219d2b2211edc747db, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:31.305539 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea6baf-d0db-4e1c-8a1d-f4c9d6529b16 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:31.306869 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ed41e-7872-4f03-809e-c2d4a00b6c3d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:31.317597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 529/1075] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:31.322986 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea6baf-d0db-4e1c-8a1d-f4c9d6529b16 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:31.380961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 547/1076] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:28:31 2022] PUT /identity/v3/projects/53a6be8ef92244f9bcfe40cf07df3f4e/users/5c2a357ac26346219d2b2211edc747db/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:28:31.391245 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:31.431541 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea6baf-d0db-4e1c-8a1d-f4c9d6529b16 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:31.469085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59e8f7-b6ca-408c-b34d-a026f0346991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:31.469893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59e8f7-b6ca-408c-b34d-a026f0346991 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:31.470177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59e8f7-b6ca-408c-b34d-a026f0346991 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:31.470452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59e8f7-b6ca-408c-b34d-a026f0346991 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:31.478717 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59e8f7-b6ca-408c-b34d-a026f0346991 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:31.480201 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59e8f7-b6ca-408c-b34d-a026f0346991 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:31.495509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-5f59e8f7-b6ca-408c-b34d-a026f0346991 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:28:31.495509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:31.495509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:31.495509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:31.495509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:31.495509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:31.495509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:31.495509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:31.495509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:31.495509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:31.495509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:31.495509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:31.495509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:28:31.495509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:28:31.495509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:28:31.496492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:28:31.496492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:31.496492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:31.496492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:31.496492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:31.496492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:31.496492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:31.496492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:31.496492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:31.496492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:31.496492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:31.496492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:31.496492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:31.496492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:31.496492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:31.496492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:31.496492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:31.497774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:28:31.497774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:28:31.497774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:28:31.497774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:28:31.497774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:28:31.497774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:28:31.497774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:28:31.497774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:31.497774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 530/1077] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:28:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:31.502741 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ed41e-7872-4f03-809e-c2d4a00b6c3d None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:31.600564 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:31.609618 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ed41e-7872-4f03-809e-c2d4a00b6c3d None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:31.643610 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9f68f-f8bb-49a3-bffc-79d44d18e7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:31.645044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9f68f-f8bb-49a3-bffc-79d44d18e7cd None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:31.645398 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9f68f-f8bb-49a3-bffc-79d44d18e7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:31.645738 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9f68f-f8bb-49a3-bffc-79d44d18e7cd None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:31.646600 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e9f68f-f8bb-49a3-bffc-79d44d18e7cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:31.646980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e9f68f-f8bb-49a3-bffc-79d44d18e7cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:31.648542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e9f68f-f8bb-49a3-bffc-79d44d18e7cd None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:31.662440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 548/1078] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:28:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:31.667892 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59e8f7-b6ca-408c-b34d-a026f0346991 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:31.770768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59e8f7-b6ca-408c-b34d-a026f0346991 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:31.797437 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:31.807525 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bc4f4-bc7d-48da-825d-b2cfc087d1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:31.808748 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bc4f4-bc7d-48da-825d-b2cfc087d1eb None admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:31.809093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bc4f4-bc7d-48da-825d-b2cfc087d1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:31.809431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bc4f4-bc7d-48da-825d-b2cfc087d1eb None admin] PATH_INFO: `/v3/projects/14265887fea245a5a37abbb3a116e352/users/159cc8befc3a4d57bbd313832084db62/roles/b1cc8597c7b144ed9b6bed9fc40f3c97` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:31.811853 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182bc4f4-bc7d-48da-825d-b2cfc087d1eb None admin] RBAC: Authorizing `identity:create_grant(project_id=14265887fea245a5a37abbb3a116e352, user_id=159cc8befc3a4d57bbd313832084db62, role_id=b1cc8597c7b144ed9b6bed9fc40f3c97)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:31.813431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182bc4f4-bc7d-48da-825d-b2cfc087d1eb None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:31.888606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 531/1079] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:28:31 2022] PUT /identity/v3/projects/14265887fea245a5a37abbb3a116e352/users/159cc8befc3a4d57bbd313832084db62/roles/b1cc8597c7b144ed9b6bed9fc40f3c97 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:28:31.964802 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fbf10-a110-48bc-9d68-74191639df67 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:31.965170 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fbf10-a110-48bc-9d68-74191639df67 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:31.965379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fbf10-a110-48bc-9d68-74191639df67 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:31.966994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 549/1080] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:31.973374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203e605-0fef-4466-b0c5-9f2ce699e63d None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:31.973790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203e605-0fef-4466-b0c5-9f2ce699e63d None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:31.973937 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203e605-0fef-4466-b0c5-9f2ce699e63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:31.991613 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-5203e605-0fef-4466-b0c5-9f2ce699e63d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:31.992873 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-5203e605-0fef-4466-b0c5-9f2ce699e63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:32.069715 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 532/1081] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:32.092382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27fbf10-a110-48bc-9d68-74191639df67 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:32.195993 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27fbf10-a110-48bc-9d68-74191639df67 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:32.230808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a216a-fce7-4012-ac93-aaeb0358d907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:32.231446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a216a-fce7-4012-ac93-aaeb0358d907 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:32.231735 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a216a-fce7-4012-ac93-aaeb0358d907 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:32.232016 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a216a-fce7-4012-ac93-aaeb0358d907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:32.253961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:32.303070 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a216a-fce7-4012-ac93-aaeb0358d907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:32.304523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a216a-fce7-4012-ac93-aaeb0358d907 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:32.407268 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 550/1082] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:28:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:28:32.413067 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5203e605-0fef-4466-b0c5-9f2ce699e63d None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:32.517646 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5203e605-0fef-4466-b0c5-9f2ce699e63d None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:32.535989 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:32.554999 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b844cc-5ac8-456e-8962-9f4063456adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:32.556046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b844cc-5ac8-456e-8962-9f4063456adb None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:32.556502 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b844cc-5ac8-456e-8962-9f4063456adb None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:32.556989 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b844cc-5ac8-456e-8962-9f4063456adb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:32.568634 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b844cc-5ac8-456e-8962-9f4063456adb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:32.570360 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b844cc-5ac8-456e-8962-9f4063456adb None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:32.589095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-b1b844cc-5ac8-456e-8962-9f4063456adb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:32.589095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:32.589095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:32.589095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:32.589095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:32.589095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:32.589095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:32.589095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:32.589095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:32.589095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:32.589095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:32.589095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:32.589095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:28:32.589095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:28:32.589095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:28:32.590284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:28:32.590284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:32.590284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:32.590284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:32.590284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:32.590284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:32.590284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:32.590284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:32.590284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:32.590284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:32.590284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:32.590284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:32.590284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:32.590284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:32.590284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:32.590284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:32.590284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:32.591380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:28:32.591380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:28:32.591380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:32.591380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:32.593240 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 533/1083] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:32.599931 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a216a-fce7-4012-ac93-aaeb0358d907 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:32.655632 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:32.702984 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a216a-fce7-4012-ac93-aaeb0358d907 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:32.738725 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475f04a-00ad-461e-b6ed-f7a2ea2b094c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:32.739624 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475f04a-00ad-461e-b6ed-f7a2ea2b094c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:32.740061 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475f04a-00ad-461e-b6ed-f7a2ea2b094c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:32.740485 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475f04a-00ad-461e-b6ed-f7a2ea2b094c None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:32.741368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475f04a-00ad-461e-b6ed-f7a2ea2b094c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:32.741819 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475f04a-00ad-461e-b6ed-f7a2ea2b094c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:32.743446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475f04a-00ad-461e-b6ed-f7a2ea2b094c None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:32.754969 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 551/1084] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:28:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:33.431637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1b09a-84b9-402c-8dd2-2a956159b8d1 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:33.432157 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1b09a-84b9-402c-8dd2-2a956159b8d1 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:33.432247 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1b09a-84b9-402c-8dd2-2a956159b8d1 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:33.433737 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 534/1085] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:33.438845 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6ce1d-6323-43d4-b25b-df54a927ba66 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:33.439262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6ce1d-6323-43d4-b25b-df54a927ba66 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:33.439416 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6ce1d-6323-43d4-b25b-df54a927ba66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:33.457686 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-c2b6ce1d-6323-43d4-b25b-df54a927ba66 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:33.458889 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-c2b6ce1d-6323-43d4-b25b-df54a927ba66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:33.524458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 552/1086] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:33.549326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1b09a-84b9-402c-8dd2-2a956159b8d1 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:33.650534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1b09a-84b9-402c-8dd2-2a956159b8d1 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:33.683582 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129774e7-f5f1-4e54-8c83-a3ce9bf372d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:33.684378 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129774e7-f5f1-4e54-8c83-a3ce9bf372d5 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:33.684694 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129774e7-f5f1-4e54-8c83-a3ce9bf372d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:33.684976 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129774e7-f5f1-4e54-8c83-a3ce9bf372d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:33.694942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129774e7-f5f1-4e54-8c83-a3ce9bf372d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:28:33.702534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-129774e7-f5f1-4e54-8c83-a3ce9bf372d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:28:33.702534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:33.702534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:33.702534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:33.702534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:33.702534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:33.702534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:33.702534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:33.702534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:33.702534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:33.702534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:33.702534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:33.702534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:28:33.702534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:28:33.702534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:28:33.702534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:28:33.703647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:28:33.703647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:28:33.703647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:28:33.703647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:28:33.703647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:33.703647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:33.703647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:33.703647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:33.703647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:33.703647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:33.703647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:33.703647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:33.703647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:33.703647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:33.703647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:33.703647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:33.704536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:33.704536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:33.704536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:33.704536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:33.704536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:28:33.704536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:28:33.704536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:28:33.704536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:28:33.704536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:28:33.704536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:28:33.704536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:28:33.704536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:33.704536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 535/1087] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:28:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:33.710291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6ce1d-6323-43d4-b25b-df54a927ba66 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:33.752059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:33.808355 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6ce1d-6323-43d4-b25b-df54a927ba66 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:33.842175 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e5a9a-ffe8-4c21-95da-f3d1d278b2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:33.842844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e5a9a-ffe8-4c21-95da-f3d1d278b2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:33.843131 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e5a9a-ffe8-4c21-95da-f3d1d278b2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:33.843410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e5a9a-ffe8-4c21-95da-f3d1d278b2b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:33.844165 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e5a9a-ffe8-4c21-95da-f3d1d278b2b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:33.844476 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e5a9a-ffe8-4c21-95da-f3d1d278b2b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:33.845875 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e5a9a-ffe8-4c21-95da-f3d1d278b2b7 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:33.853824 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 553/1088] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:28:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:33.859467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129774e7-f5f1-4e54-8c83-a3ce9bf372d5 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:33.887997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:33.917968 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57e2e6-867a-490a-8c43-c9a707ec42ad None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:33.918262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57e2e6-867a-490a-8c43-c9a707ec42ad None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:33.918546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57e2e6-867a-490a-8c43-c9a707ec42ad None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:33.919921 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 554/1089] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:28:33.924658 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025e8b8-6e6e-4688-8a50-1cd11ff66044 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:33.924951 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025e8b8-6e6e-4688-8a50-1cd11ff66044 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:33.925356 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025e8b8-6e6e-4688-8a50-1cd11ff66044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:33.941985 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-5025e8b8-6e6e-4688-8a50-1cd11ff66044 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:33.943157 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-5025e8b8-6e6e-4688-8a50-1cd11ff66044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:33.961428 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129774e7-f5f1-4e54-8c83-a3ce9bf372d5 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:33.994377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ca8a8-0d55-434c-9e63-cd15c1fc6522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:33.995248 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ca8a8-0d55-434c-9e63-cd15c1fc6522 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:33.995549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ca8a8-0d55-434c-9e63-cd15c1fc6522 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:33.995808 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ca8a8-0d55-434c-9e63-cd15c1fc6522 None admin] PATH_INFO: `/v3/users/5c2a357ac26346219d2b2211edc747db` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:33.997015 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ca8a8-0d55-434c-9e63-cd15c1fc6522 None admin] RBAC: Authorizing `identity:get_user(user_id=5c2a357ac26346219d2b2211edc747db)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:33.998441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ca8a8-0d55-434c-9e63-cd15c1fc6522 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:34.000068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 536/1090] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:28:33 2022] GET /identity/v3/users/5c2a357ac26346219d2b2211edc747db => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:34.006152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ca8a8-0d55-434c-9e63-cd15c1fc6522 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:34.007777 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 555/1091] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:28:34.031434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025e8b8-6e6e-4688-8a50-1cd11ff66044 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:34.064180 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:34.098362 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:34.103980 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ca8a8-0d55-434c-9e63-cd15c1fc6522 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:34.129576 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025e8b8-6e6e-4688-8a50-1cd11ff66044 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:34.138085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980c77f-0806-4fce-9d35-230e9e311755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:34.138826 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980c77f-0806-4fce-9d35-230e9e311755 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:34.139113 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980c77f-0806-4fce-9d35-230e9e311755 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:34.139388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980c77f-0806-4fce-9d35-230e9e311755 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:34.162534 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9e40e-9873-4854-ac9d-4c80342a5a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:34.163342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9e40e-9873-4854-ac9d-4c80342a5a49 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:34.163628 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9e40e-9873-4854-ac9d-4c80342a5a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:34.163903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9e40e-9873-4854-ac9d-4c80342a5a49 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:34.174091 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9e40e-9873-4854-ac9d-4c80342a5a49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:28:34.182994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-dbe9e40e-9873-4854-ac9d-4c80342a5a49 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:28:34.182994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:34.182994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:34.182994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:34.182994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:34.182994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:34.182994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:34.182994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:34.182994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:34.182994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:34.182994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:34.182994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:34.182994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:28:34.182994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:28:34.182994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:28:34.182994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:28:34.184101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:28:34.184101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:28:34.184101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:28:34.184101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:28:34.184101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:34.184101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:34.184101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:34.184101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:34.184101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:34.184101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:34.184101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:34.184101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:34.184101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:34.184101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:34.184101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:34.184101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:34.185216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:34.185216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:34.185216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:34.185216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:34.185216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:28:34.185216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:28:34.185216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:28:34.185216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:28:34.185216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:28:34.185216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:28:34.185216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:28:34.185216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:34.185216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 556/1092] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 23 22:28:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:34.190137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9e40e-9873-4854-ac9d-4c80342a5a49 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:34.208200 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0980c77f-0806-4fce-9d35-230e9e311755 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:34.209764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0980c77f-0806-4fce-9d35-230e9e311755 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:34.281788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:34.289537 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9e40e-9873-4854-ac9d-4c80342a5a49 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:34.302961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:34.309498 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 537/1093] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:28:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 22:28:34.314980 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980c77f-0806-4fce-9d35-230e9e311755 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:34.324398 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2cc10-e6d2-4595-aaa3-0479990c57d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:34.325152 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2cc10-e6d2-4595-aaa3-0479990c57d1 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:34.338652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2cc10-e6d2-4595-aaa3-0479990c57d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:34.338652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2cc10-e6d2-4595-aaa3-0479990c57d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:34.338652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2cc10-e6d2-4595-aaa3-0479990c57d1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:34.338652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2cc10-e6d2-4595-aaa3-0479990c57d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:34.338652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2cc10-e6d2-4595-aaa3-0479990c57d1 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:34.338652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 557/1094] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 23 22:28:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:34.341792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2cc10-e6d2-4595-aaa3-0479990c57d1 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:34.415003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980c77f-0806-4fce-9d35-230e9e311755 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:34.441156 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2cc10-e6d2-4595-aaa3-0479990c57d1 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:34.449858 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32614587-7d3e-4b18-a769-7201587ecc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:34.450747 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32614587-7d3e-4b18-a769-7201587ecc5d None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:34.451030 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32614587-7d3e-4b18-a769-7201587ecc5d None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:34.451305 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32614587-7d3e-4b18-a769-7201587ecc5d None admin] PATH_INFO: `/v3/projects/53a6be8ef92244f9bcfe40cf07df3f4e` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:34.452475 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32614587-7d3e-4b18-a769-7201587ecc5d None admin] RBAC: Authorizing `identity:get_project(project_id=53a6be8ef92244f9bcfe40cf07df3f4e)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:34.453974 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32614587-7d3e-4b18-a769-7201587ecc5d None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:34.455667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 538/1095] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:28:34 2022] GET /identity/v3/projects/53a6be8ef92244f9bcfe40cf07df3f4e => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:34.462095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32614587-7d3e-4b18-a769-7201587ecc5d None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:34.479749 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63941a3-e44d-47ed-a7c8-e7d9e1859305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:34.480545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63941a3-e44d-47ed-a7c8-e7d9e1859305 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:34.480865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63941a3-e44d-47ed-a7c8-e7d9e1859305 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:34.481149 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63941a3-e44d-47ed-a7c8-e7d9e1859305 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:34.492534 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63941a3-e44d-47ed-a7c8-e7d9e1859305 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:34.494050 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63941a3-e44d-47ed-a7c8-e7d9e1859305 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:34.511473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-a63941a3-e44d-47ed-a7c8-e7d9e1859305 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:34.511473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:34.511473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:34.511473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:34.511473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:34.511473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:34.511473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:34.511473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:34.511473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:34.511473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:34.511473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:34.511473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:34.511473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:28:34.511473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:28:34.511473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:28:34.512620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:28:34.512620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:34.512620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:34.512620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:34.512620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:34.512620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:34.512620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:34.512620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:34.512620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:34.512620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:34.512620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:34.512620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:34.512620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:34.512620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:34.512620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:34.512620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:34.512620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:34.513710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:28:34.513710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:28:34.513710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:34.513710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:34.514312 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 558/1096] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:34.518156 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63941a3-e44d-47ed-a7c8-e7d9e1859305 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:34.522589 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:34.529534 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:34.563570 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32614587-7d3e-4b18-a769-7201587ecc5d None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:34.596780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb7e49-7161-4be2-aec6-86b1f78df99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:34.597938 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb7e49-7161-4be2-aec6-86b1f78df99c None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:34.598214 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb7e49-7161-4be2-aec6-86b1f78df99c None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:34.598484 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb7e49-7161-4be2-aec6-86b1f78df99c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:34.599302 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb7e49-7161-4be2-aec6-86b1f78df99c None admin] RBAC: Adding query filter params (role.id=6ff7708b59984b9287cc81d46a57b4b4, scope.project.id=53a6be8ef92244f9bcfe40cf07df3f4e, user.id=5c2a357ac26346219d2b2211edc747db) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:34.599611 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb7e49-7161-4be2-aec6-86b1f78df99c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:34.601069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb7e49-7161-4be2-aec6-86b1f78df99c None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:34.606755 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 539/1097] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:28:34 2022] GET /identity/v3/role_assignments?user.id=5c2a357ac26346219d2b2211edc747db&scope.project.id=53a6be8ef92244f9bcfe40cf07df3f4e&role.id=6ff7708b59984b9287cc81d46a57b4b4 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:34.618914 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63941a3-e44d-47ed-a7c8-e7d9e1859305 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:34.655375 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08a4c0-6787-4975-b72f-5fb66f1f96e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:34.656063 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08a4c0-6787-4975-b72f-5fb66f1f96e7 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:34.656348 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08a4c0-6787-4975-b72f-5fb66f1f96e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:34.656663 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08a4c0-6787-4975-b72f-5fb66f1f96e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:34.657408 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d08a4c0-6787-4975-b72f-5fb66f1f96e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:34.657712 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d08a4c0-6787-4975-b72f-5fb66f1f96e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:34.659124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d08a4c0-6787-4975-b72f-5fb66f1f96e7 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:34.673161 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 559/1098] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:28:34.679952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb7e49-7161-4be2-aec6-86b1f78df99c None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:34.785809 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb7e49-7161-4be2-aec6-86b1f78df99c None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:34.811991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:34.820407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba22f18-2ddd-434e-9d5a-d67c4b6290b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:34.821253 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba22f18-2ddd-434e-9d5a-d67c4b6290b8 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:34.821538 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba22f18-2ddd-434e-9d5a-d67c4b6290b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:34.821816 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba22f18-2ddd-434e-9d5a-d67c4b6290b8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:34.829530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba22f18-2ddd-434e-9d5a-d67c4b6290b8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:34.831112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba22f18-2ddd-434e-9d5a-d67c4b6290b8 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:34.845827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-0ba22f18-2ddd-434e-9d5a-d67c4b6290b8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:28:34.845827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:34.845827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:34.845827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:34.845827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:34.845827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:34.845827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:34.845827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:34.845827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:34.845827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:34.845827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:34.845827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:34.845827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:28:34.845827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:28:34.845827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:28:34.845827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:28:34.847041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:34.847041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:34.847041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:28:34.847041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:28:34.847041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:28:34.847041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:28:34.847041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:34.847041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:34.847041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:34.847041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:34.847041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:34.847041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:34.847041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:34.847041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:34.847041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:34.847041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:34.847041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:34.847041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:34.848045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:34.848045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:34.848045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:28:34.848045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:28:34.848045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:28:34.848045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:28:34.848045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:28:34.848045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:28:34.848045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:28:34.848045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:34.848045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 540/1099] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 23 22:28:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:34.853231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08a4c0-6787-4975-b72f-5fb66f1f96e7 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:34.919418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:34.953984 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08a4c0-6787-4975-b72f-5fb66f1f96e7 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:34.986931 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad3cbc-2388-49c7-9dc3-67f796f5f987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:34.988114 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad3cbc-2388-49c7-9dc3-67f796f5f987 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:34.988428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad3cbc-2388-49c7-9dc3-67f796f5f987 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:34.988733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad3cbc-2388-49c7-9dc3-67f796f5f987 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:34.989567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad3cbc-2388-49c7-9dc3-67f796f5f987 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:34.989875 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad3cbc-2388-49c7-9dc3-67f796f5f987 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:34.991294 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad3cbc-2388-49c7-9dc3-67f796f5f987 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:35.008956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 560/1100] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 23 22:28:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:35.014683 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba22f18-2ddd-434e-9d5a-d67c4b6290b8 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:35.113885 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba22f18-2ddd-434e-9d5a-d67c4b6290b8 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:35.146711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba03f9-1654-443c-90b1-2f0d4a1fe616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:35.147363 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba03f9-1654-443c-90b1-2f0d4a1fe616 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:35.147650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba03f9-1654-443c-90b1-2f0d4a1fe616 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:35.147930 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba03f9-1654-443c-90b1-2f0d4a1fe616 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:35.202236 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:35.214302 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba03f9-1654-443c-90b1-2f0d4a1fe616 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:35.215756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba03f9-1654-443c-90b1-2f0d4a1fe616 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:35.318383 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 541/1101] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:28:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:28:35.324179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad3cbc-2388-49c7-9dc3-67f796f5f987 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:35.424189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad3cbc-2388-49c7-9dc3-67f796f5f987 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:35.449599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:35.458424 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b0577-5313-4071-a283-d24511fc7538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:35.467326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b0577-5313-4071-a283-d24511fc7538 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:35.467326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b0577-5313-4071-a283-d24511fc7538 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:35.467326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b0577-5313-4071-a283-d24511fc7538 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:35.469494 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b0577-5313-4071-a283-d24511fc7538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:35.470943 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b0577-5313-4071-a283-d24511fc7538 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:35.489637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-133b0577-5313-4071-a283-d24511fc7538 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:35.489637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:35.489637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:35.489637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:35.489637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:35.489637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:35.489637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:35.489637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:35.489637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:35.489637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:35.489637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:35.489637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:35.489637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:28:35.489637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:28:35.489637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:28:35.490621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:28:35.490621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:35.490621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:35.490621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:35.490621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:35.490621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:35.490621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:35.490621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:35.490621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:35.490621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:35.490621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:35.490621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:35.490621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:35.490621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:35.490621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:35.490621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:35.490621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:35.491511 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:28:35.491511 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:28:35.491511 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:28:35.491511 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:35.492717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 561/1102] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:28:35.496588 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba03f9-1654-443c-90b1-2f0d4a1fe616 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:35.580399 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:35.593563 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba03f9-1654-443c-90b1-2f0d4a1fe616 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:35.628144 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aac617-0692-46a3-85a6-4e785a18e1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:35.628954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aac617-0692-46a3-85a6-4e785a18e1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:35.629236 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aac617-0692-46a3-85a6-4e785a18e1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:35.629513 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aac617-0692-46a3-85a6-4e785a18e1c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:35.630232 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aac617-0692-46a3-85a6-4e785a18e1c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:35.630542 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aac617-0692-46a3-85a6-4e785a18e1c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:35.631906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aac617-0692-46a3-85a6-4e785a18e1c7 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:35.642805 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 542/1103] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:35.649221 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b0577-5313-4071-a283-d24511fc7538 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:35.749401 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b0577-5313-4071-a283-d24511fc7538 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:35.784478 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837dde01-04b2-4ab2-9558-1b6b867b1001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:35.785813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dde01-04b2-4ab2-9558-1b6b867b1001 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:35.786339 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dde01-04b2-4ab2-9558-1b6b867b1001 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:35.786719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dde01-04b2-4ab2-9558-1b6b867b1001 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:35.793755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837dde01-04b2-4ab2-9558-1b6b867b1001 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:35.795194 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837dde01-04b2-4ab2-9558-1b6b867b1001 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:35.809428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-837dde01-04b2-4ab2-9558-1b6b867b1001 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:28:35.809428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:35.809428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:35.809428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:35.809428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:35.809428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:35.809428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:35.809428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:35.809428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:35.809428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:35.809428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:35.809428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:35.809428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:28:35.809428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:28:35.809428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:28:35.810490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:28:35.810490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:35.810490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:35.810490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:35.810490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:35.810490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:35.810490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:35.810490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:35.810490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:35.810490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:35.810490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:35.810490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:35.810490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:35.810490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:35.810490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:35.810490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:35.810490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:35.811799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:28:35.811799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:28:35.811799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:28:35.811799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:28:35.811799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:28:35.811799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:28:35.811799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:28:35.811799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:35.811799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 562/1104] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:28:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:35.817396 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aac617-0692-46a3-85a6-4e785a18e1c7 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:35.915088 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aac617-0692-46a3-85a6-4e785a18e1c7 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:35.948517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee2d51-21d7-4882-8320-b058a62b57dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:35.949200 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee2d51-21d7-4882-8320-b058a62b57dc None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:35.949491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee2d51-21d7-4882-8320-b058a62b57dc None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:35.949774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee2d51-21d7-4882-8320-b058a62b57dc None admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:35.950560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee2d51-21d7-4882-8320-b058a62b57dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:35.950878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee2d51-21d7-4882-8320-b058a62b57dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:35.952312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee2d51-21d7-4882-8320-b058a62b57dc None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:35.964163 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 543/1105] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:28:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:35.970088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837dde01-04b2-4ab2-9558-1b6b867b1001 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:36.012486 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:36.071798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837dde01-04b2-4ab2-9558-1b6b867b1001 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:36.104948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302c09e-e5ef-4576-910b-0a3c860891ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:36.105619 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302c09e-e5ef-4576-910b-0a3c860891ae None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:36.105942 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302c09e-e5ef-4576-910b-0a3c860891ae None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:36.106255 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302c09e-e5ef-4576-910b-0a3c860891ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:36.107103 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302c09e-e5ef-4576-910b-0a3c860891ae None admin] RBAC: Adding query filter params (role.id=b1cc8597c7b144ed9b6bed9fc40f3c97, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=159cc8befc3a4d57bbd313832084db62) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:36.107428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302c09e-e5ef-4576-910b-0a3c860891ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:36.108902 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302c09e-e5ef-4576-910b-0a3c860891ae None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:36.114849 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 563/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:28:35 2022] GET /identity/v3/role_assignments?user.id=159cc8befc3a4d57bbd313832084db62&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=b1cc8597c7b144ed9b6bed9fc40f3c97 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:36.681014 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfaba3a-a6a6-422c-a4ee-eec9c72f3dc9 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:36.681503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfaba3a-a6a6-422c-a4ee-eec9c72f3dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:36.681503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfaba3a-a6a6-422c-a4ee-eec9c72f3dc9 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:36.682878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 544/1107] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:36.688203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fd166-621d-45a6-8b6e-79d88326e75d None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:36.688695 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fd166-621d-45a6-8b6e-79d88326e75d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:36.688914 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fd166-621d-45a6-8b6e-79d88326e75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:36.706665 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-e88fd166-621d-45a6-8b6e-79d88326e75d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:36.707927 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e88fd166-621d-45a6-8b6e-79d88326e75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:36.778458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 564/1108] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:36.804923 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfaba3a-a6a6-422c-a4ee-eec9c72f3dc9 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:36.819572 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:36.912101 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfaba3a-a6a6-422c-a4ee-eec9c72f3dc9 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:36.946988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca209b-b199-44a3-b301-383637200a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:36.947715 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca209b-b199-44a3-b301-383637200a67 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:36.947992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca209b-b199-44a3-b301-383637200a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:36.948265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca209b-b199-44a3-b301-383637200a67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:36.949473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca209b-b199-44a3-b301-383637200a67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:36.950901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca209b-b199-44a3-b301-383637200a67 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:36.952518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 545/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:36.959056 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fd166-621d-45a6-8b6e-79d88326e75d None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:36.990193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:37.057855 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fd166-621d-45a6-8b6e-79d88326e75d None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:37.089947 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630d1d6-0085-43cd-bbde-6b4332cf6a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:37.090619 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630d1d6-0085-43cd-bbde-6b4332cf6a52 None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:37.090940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630d1d6-0085-43cd-bbde-6b4332cf6a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:37.091255 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630d1d6-0085-43cd-bbde-6b4332cf6a52 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:37.091991 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630d1d6-0085-43cd-bbde-6b4332cf6a52 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:37.093440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630d1d6-0085-43cd-bbde-6b4332cf6a52 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:37.204247 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 565/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 22:28:36 2022] POST /identity/v3/users => generated 286 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:28:39.245303 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3693296-90b8-40aa-835c-96ef57ed5d61 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:39.245864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3693296-90b8-40aa-835c-96ef57ed5d61 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:39.245864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3693296-90b8-40aa-835c-96ef57ed5d61 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:39.247210 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 546/1111] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:39.252312 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a123b-28e3-49d6-b0a2-ef4c88ea40be None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:39.252762 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a123b-28e3-49d6-b0a2-ef4c88ea40be None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:39.253015 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a123b-28e3-49d6-b0a2-ef4c88ea40be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:39.269544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-e15a123b-28e3-49d6-b0a2-ef4c88ea40be None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:39.270725 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e15a123b-28e3-49d6-b0a2-ef4c88ea40be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:39.336334 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 566/1112] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:39.360847 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3693296-90b8-40aa-835c-96ef57ed5d61 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:39.437233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:39.460385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3693296-90b8-40aa-835c-96ef57ed5d61 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:39.493732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38fa9b-d6f6-4383-9493-d95cc6e31a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:39.494578 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38fa9b-d6f6-4383-9493-d95cc6e31a07 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:39.494859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38fa9b-d6f6-4383-9493-d95cc6e31a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:39.495132 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38fa9b-d6f6-4383-9493-d95cc6e31a07 None admin] PATH_INFO: `/v3/roles/c3ad3cf6be504e35aaec10b7b9f3f34d` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:39.496656 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38fa9b-d6f6-4383-9493-d95cc6e31a07 None admin] RBAC: Authorizing `identity:get_role(role_id=c3ad3cf6be504e35aaec10b7b9f3f34d)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:39.498238 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38fa9b-d6f6-4383-9493-d95cc6e31a07 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:39.499637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 547/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:28:39 2022] GET /identity/v3/roles/c3ad3cf6be504e35aaec10b7b9f3f34d => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:39.505618 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a123b-28e3-49d6-b0a2-ef4c88ea40be None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:39.524082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:39.604999 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a123b-28e3-49d6-b0a2-ef4c88ea40be None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:39.640042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368abc5-848a-4023-a776-a3397c806c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:39.640952 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368abc5-848a-4023-a776-a3397c806c10 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:39.641277 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368abc5-848a-4023-a776-a3397c806c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:39.641581 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368abc5-848a-4023-a776-a3397c806c10 None admin] PATH_INFO: `/v3/users/d181aad33cd446668560e51bb39c973c` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:39.654776 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368abc5-848a-4023-a776-a3397c806c10 None admin] RBAC: Authorizing `identity:get_user(user_id=d181aad33cd446668560e51bb39c973c)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:39.656245 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368abc5-848a-4023-a776-a3397c806c10 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:39.657927 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 567/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:28:39 2022] GET /identity/v3/users/d181aad33cd446668560e51bb39c973c => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:39.664616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38fa9b-d6f6-4383-9493-d95cc6e31a07 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:39.762971 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38fa9b-d6f6-4383-9493-d95cc6e31a07 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:39.797470 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52ae9d-361a-4b0f-b68d-08e72187ed7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:39.798104 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52ae9d-361a-4b0f-b68d-08e72187ed7c None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:39.798389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52ae9d-361a-4b0f-b68d-08e72187ed7c None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:39.798667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52ae9d-361a-4b0f-b68d-08e72187ed7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:39.812629 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:39.864691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52ae9d-361a-4b0f-b68d-08e72187ed7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:39.866261 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52ae9d-361a-4b0f-b68d-08e72187ed7c None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:39.961706 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 548/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:28:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:28:39.967540 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368abc5-848a-4023-a776-a3397c806c10 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:40.043307 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:40.065988 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368abc5-848a-4023-a776-a3397c806c10 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:40.098878 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9a11f-884f-4a1e-a0ba-ae2276c8f4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:40.099728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9a11f-884f-4a1e-a0ba-ae2276c8f4cd None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:40.100014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9a11f-884f-4a1e-a0ba-ae2276c8f4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:40.100291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9a11f-884f-4a1e-a0ba-ae2276c8f4cd None admin] PATH_INFO: `/v3/projects/53a6be8ef92244f9bcfe40cf07df3f4e` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:40.101518 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9a11f-884f-4a1e-a0ba-ae2276c8f4cd None admin] RBAC: Authorizing `identity:get_project(project_id=53a6be8ef92244f9bcfe40cf07df3f4e)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:40.102938 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9a11f-884f-4a1e-a0ba-ae2276c8f4cd None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:40.104646 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 568/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:28:39 2022] GET /identity/v3/projects/53a6be8ef92244f9bcfe40cf07df3f4e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:40.110532 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52ae9d-361a-4b0f-b68d-08e72187ed7c None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:40.155116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:40.209386 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52ae9d-361a-4b0f-b68d-08e72187ed7c None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:40.241642 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1989ef-b66b-4d5f-867e-adbd7d41d60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:40.242291 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1989ef-b66b-4d5f-867e-adbd7d41d60c None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:40.242581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1989ef-b66b-4d5f-867e-adbd7d41d60c None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:40.242865 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1989ef-b66b-4d5f-867e-adbd7d41d60c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:40.243692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1989ef-b66b-4d5f-867e-adbd7d41d60c None admin] RBAC: Adding query filter params (role.id=c3ad3cf6be504e35aaec10b7b9f3f34d, scope.project.id=53a6be8ef92244f9bcfe40cf07df3f4e, user.id=d181aad33cd446668560e51bb39c973c) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:40.244906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1989ef-b66b-4d5f-867e-adbd7d41d60c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:40.245543 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1989ef-b66b-4d5f-867e-adbd7d41d60c None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:40.250912 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 549/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:28:40 2022] GET /identity/v3/role_assignments?user.id=d181aad33cd446668560e51bb39c973c&scope.project.id=53a6be8ef92244f9bcfe40cf07df3f4e&role.id=c3ad3cf6be504e35aaec10b7b9f3f34d => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:42.238709 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f9063-c8d1-477a-baa3-856b214dea34 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:42.239268 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f9063-c8d1-477a-baa3-856b214dea34 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:42.239268 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f9063-c8d1-477a-baa3-856b214dea34 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:42.240655 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 569/1118] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:42.245749 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37754bc9-e03a-4bec-bb60-095fda67f16f None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:42.246169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37754bc9-e03a-4bec-bb60-095fda67f16f None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:42.246306 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37754bc9-e03a-4bec-bb60-095fda67f16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:42.262720 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-37754bc9-e03a-4bec-bb60-095fda67f16f None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:42.263887 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-37754bc9-e03a-4bec-bb60-095fda67f16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:42.329394 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 550/1119] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:42.352703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f9063-c8d1-477a-baa3-856b214dea34 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:42.452208 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f9063-c8d1-477a-baa3-856b214dea34 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:42.485490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb472c-5fa4-405a-a143-2b3a695bb99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:42.486336 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb472c-5fa4-405a-a143-2b3a695bb99b None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:42.486622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb472c-5fa4-405a-a143-2b3a695bb99b None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:42.486900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb472c-5fa4-405a-a143-2b3a695bb99b None admin] PATH_INFO: `/v3/roles/c3ad3cf6be504e35aaec10b7b9f3f34d` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:42.488387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb472c-5fa4-405a-a143-2b3a695bb99b None admin] RBAC: Authorizing `identity:get_role(role_id=c3ad3cf6be504e35aaec10b7b9f3f34d)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:42.489830 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb472c-5fa4-405a-a143-2b3a695bb99b None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:42.491132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 570/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:28:42 2022] GET /identity/v3/roles/c3ad3cf6be504e35aaec10b7b9f3f34d => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:42.496872 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37754bc9-e03a-4bec-bb60-095fda67f16f None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:42.535288 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:42.593280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37754bc9-e03a-4bec-bb60-095fda67f16f None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:42.626282 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e2154-42dc-4733-9dee-88b0c3c5cf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:42.627140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e2154-42dc-4733-9dee-88b0c3c5cf47 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:42.627427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e2154-42dc-4733-9dee-88b0c3c5cf47 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:42.627704 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e2154-42dc-4733-9dee-88b0c3c5cf47 None admin] PATH_INFO: `/v3/users/d181aad33cd446668560e51bb39c973c` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:42.629099 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e2154-42dc-4733-9dee-88b0c3c5cf47 None admin] RBAC: Authorizing `identity:get_user(user_id=d181aad33cd446668560e51bb39c973c)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:42.630619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e2154-42dc-4733-9dee-88b0c3c5cf47 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:42.632308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 551/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:28:42 2022] GET /identity/v3/users/d181aad33cd446668560e51bb39c973c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:42.638917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb472c-5fa4-405a-a143-2b3a695bb99b None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:42.656557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:42.742265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb472c-5fa4-405a-a143-2b3a695bb99b None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:42.775553 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f6fca-7c0d-42df-a5e1-0bdd1eb444fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:42.776404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f6fca-7c0d-42df-a5e1-0bdd1eb444fb None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:42.776714 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f6fca-7c0d-42df-a5e1-0bdd1eb444fb None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:42.776992 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f6fca-7c0d-42df-a5e1-0bdd1eb444fb None admin] PATH_INFO: `/v3/projects/53a6be8ef92244f9bcfe40cf07df3f4e` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:42.778153 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f6fca-7c0d-42df-a5e1-0bdd1eb444fb None admin] RBAC: Authorizing `identity:get_project(project_id=53a6be8ef92244f9bcfe40cf07df3f4e)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:42.779573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f6fca-7c0d-42df-a5e1-0bdd1eb444fb None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:42.781471 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 571/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:28:42 2022] GET /identity/v3/projects/53a6be8ef92244f9bcfe40cf07df3f4e => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:42.787526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e2154-42dc-4733-9dee-88b0c3c5cf47 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:42.887885 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e2154-42dc-4733-9dee-88b0c3c5cf47 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:42.895830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:42.921387 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617a611-4244-4bf3-92d0-8548a69d16a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:42.922260 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617a611-4244-4bf3-92d0-8548a69d16a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:42.922544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617a611-4244-4bf3-92d0-8548a69d16a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:42.922821 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617a611-4244-4bf3-92d0-8548a69d16a2 None admin] PATH_INFO: `/v3/projects/53a6be8ef92244f9bcfe40cf07df3f4e/users/d181aad33cd446668560e51bb39c973c/roles/c3ad3cf6be504e35aaec10b7b9f3f34d` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:42.925041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617a611-4244-4bf3-92d0-8548a69d16a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=53a6be8ef92244f9bcfe40cf07df3f4e, user_id=d181aad33cd446668560e51bb39c973c, role_id=c3ad3cf6be504e35aaec10b7b9f3f34d)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:42.926542 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617a611-4244-4bf3-92d0-8548a69d16a2 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:42.974759 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 552/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:28:42 2022] PUT /identity/v3/projects/53a6be8ef92244f9bcfe40cf07df3f4e/users/d181aad33cd446668560e51bb39c973c/roles/c3ad3cf6be504e35aaec10b7b9f3f34d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:28:44.979624 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f305918-f671-447e-b912-53e461a85ef9 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:44.980180 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f305918-f671-447e-b912-53e461a85ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:44.980180 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f305918-f671-447e-b912-53e461a85ef9 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:44.981588 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 572/1124] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:44.986816 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063623be-5698-4216-86b4-fe494a3aedb0 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:44.987260 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063623be-5698-4216-86b4-fe494a3aedb0 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:44.987359 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063623be-5698-4216-86b4-fe494a3aedb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:45.004435 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-063623be-5698-4216-86b4-fe494a3aedb0 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:45.005639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-063623be-5698-4216-86b4-fe494a3aedb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:45.069608 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 553/1125] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:45.093796 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f305918-f671-447e-b912-53e461a85ef9 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:45.173112 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:45.192494 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f305918-f671-447e-b912-53e461a85ef9 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:45.225183 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd148aa-41e3-4f5a-bd5d-847edd76ff8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:45.226035 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd148aa-41e3-4f5a-bd5d-847edd76ff8d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:45.226322 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd148aa-41e3-4f5a-bd5d-847edd76ff8d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:45.226601 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd148aa-41e3-4f5a-bd5d-847edd76ff8d None admin] PATH_INFO: `/v3/roles/c3ad3cf6be504e35aaec10b7b9f3f34d` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:45.228065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd148aa-41e3-4f5a-bd5d-847edd76ff8d None admin] RBAC: Authorizing `identity:get_role(role_id=c3ad3cf6be504e35aaec10b7b9f3f34d)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:45.229453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd148aa-41e3-4f5a-bd5d-847edd76ff8d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:45.230796 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 573/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:28:45 2022] GET /identity/v3/roles/c3ad3cf6be504e35aaec10b7b9f3f34d => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:45.236438 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063623be-5698-4216-86b4-fe494a3aedb0 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:45.327301 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:45.333706 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063623be-5698-4216-86b4-fe494a3aedb0 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:45.367228 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c268a958-3214-4d77-8b6f-03b4b0a266c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:45.368266 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268a958-3214-4d77-8b6f-03b4b0a266c0 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:45.368555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268a958-3214-4d77-8b6f-03b4b0a266c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:45.368863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268a958-3214-4d77-8b6f-03b4b0a266c0 None admin] PATH_INFO: `/v3/users/d181aad33cd446668560e51bb39c973c` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:45.370023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c268a958-3214-4d77-8b6f-03b4b0a266c0 None admin] RBAC: Authorizing `identity:get_user(user_id=d181aad33cd446668560e51bb39c973c)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:45.371452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c268a958-3214-4d77-8b6f-03b4b0a266c0 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:45.373112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 554/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:28:45 2022] GET /identity/v3/users/d181aad33cd446668560e51bb39c973c => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:45.379476 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd148aa-41e3-4f5a-bd5d-847edd76ff8d None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:45.475880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd148aa-41e3-4f5a-bd5d-847edd76ff8d None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:45.508963 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebf792-edfe-45a6-a8d3-45f944e4ca55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:45.509527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebf792-edfe-45a6-a8d3-45f944e4ca55 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:45.509812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebf792-edfe-45a6-a8d3-45f944e4ca55 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:45.510085 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebf792-edfe-45a6-a8d3-45f944e4ca55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:45.526851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:45.575797 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ebf792-edfe-45a6-a8d3-45f944e4ca55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:45.577269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ebf792-edfe-45a6-a8d3-45f944e4ca55 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:45.671220 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 574/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:28:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:28:45.677684 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c268a958-3214-4d77-8b6f-03b4b0a266c0 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:45.774366 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c268a958-3214-4d77-8b6f-03b4b0a266c0 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:45.807586 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31632303-b4c7-410c-a3fb-95ba1c4015dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:45.808430 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31632303-b4c7-410c-a3fb-95ba1c4015dd None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:45.808748 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31632303-b4c7-410c-a3fb-95ba1c4015dd None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:45.809028 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31632303-b4c7-410c-a3fb-95ba1c4015dd None admin] PATH_INFO: `/v3/projects/53a6be8ef92244f9bcfe40cf07df3f4e` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:45.810220 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31632303-b4c7-410c-a3fb-95ba1c4015dd None admin] RBAC: Authorizing `identity:get_project(project_id=53a6be8ef92244f9bcfe40cf07df3f4e)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:45.811638 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31632303-b4c7-410c-a3fb-95ba1c4015dd None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:45.813344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 555/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:28:45 2022] GET /identity/v3/projects/53a6be8ef92244f9bcfe40cf07df3f4e => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:45.819287 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebf792-edfe-45a6-a8d3-45f944e4ca55 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:45.866580 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:45.915210 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebf792-edfe-45a6-a8d3-45f944e4ca55 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:45.948138 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b622b97-dbce-416a-9052-0ac20b74a8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:45.948810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b622b97-dbce-416a-9052-0ac20b74a8fe None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:45.949089 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b622b97-dbce-416a-9052-0ac20b74a8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:45.949361 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b622b97-dbce-416a-9052-0ac20b74a8fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:45.950172 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b622b97-dbce-416a-9052-0ac20b74a8fe None admin] RBAC: Adding query filter params (role.id=c3ad3cf6be504e35aaec10b7b9f3f34d, scope.project.id=53a6be8ef92244f9bcfe40cf07df3f4e, user.id=d181aad33cd446668560e51bb39c973c) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:45.950478 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b622b97-dbce-416a-9052-0ac20b74a8fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:45.951876 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b622b97-dbce-416a-9052-0ac20b74a8fe None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:45.957554 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 575/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:28:45 2022] GET /identity/v3/role_assignments?user.id=d181aad33cd446668560e51bb39c973c&scope.project.id=53a6be8ef92244f9bcfe40cf07df3f4e&role.id=c3ad3cf6be504e35aaec10b7b9f3f34d => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:48.003713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02eaf53-f4ea-4e3f-a25f-8809ee3171e3 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:48.004272 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02eaf53-f4ea-4e3f-a25f-8809ee3171e3 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:48.004272 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02eaf53-f4ea-4e3f-a25f-8809ee3171e3 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:48.005675 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 556/1131] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:48.010813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a9551-e5cd-4c1e-8ea3-3ce6388e2d8b None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:48.011249 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a9551-e5cd-4c1e-8ea3-3ce6388e2d8b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:48.011350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a9551-e5cd-4c1e-8ea3-3ce6388e2d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:48.028234 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-460a9551-e5cd-4c1e-8ea3-3ce6388e2d8b None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:48.029487 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-460a9551-e5cd-4c1e-8ea3-3ce6388e2d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:48.093501 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 576/1132] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:48.117005 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02eaf53-f4ea-4e3f-a25f-8809ee3171e3 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:48.148021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:48.214646 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02eaf53-f4ea-4e3f-a25f-8809ee3171e3 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:48.246221 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3e499-24a4-4512-9291-0e76bd373f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:48.246993 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3e499-24a4-4512-9291-0e76bd373f52 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:48.247276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3e499-24a4-4512-9291-0e76bd373f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:48.247551 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3e499-24a4-4512-9291-0e76bd373f52 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:48.248722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b3e499-24a4-4512-9291-0e76bd373f52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:48.250141 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b3e499-24a4-4512-9291-0e76bd373f52 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:48.251714 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 557/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:48.258014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a9551-e5cd-4c1e-8ea3-3ce6388e2d8b None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:48.301274 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:48.357582 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a9551-e5cd-4c1e-8ea3-3ce6388e2d8b None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:48.391006 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204c78f-3d26-4eaf-90cd-46fc0aa1faca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:48.391663 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204c78f-3d26-4eaf-90cd-46fc0aa1faca None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:48.391950 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204c78f-3d26-4eaf-90cd-46fc0aa1faca None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:48.392230 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204c78f-3d26-4eaf-90cd-46fc0aa1faca None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:48.392972 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204c78f-3d26-4eaf-90cd-46fc0aa1faca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:48.395062 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204c78f-3d26-4eaf-90cd-46fc0aa1faca None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:48.495020 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 577/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 22:28:48 2022] POST /identity/v3/projects => generated 311 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:28:50.551073 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d2886-719e-44e1-9d7f-fd0bbbe89973 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:50.551576 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d2886-719e-44e1-9d7f-fd0bbbe89973 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:50.551576 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d2886-719e-44e1-9d7f-fd0bbbe89973 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:50.552948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 558/1135] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:50.558215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a44cb-394e-4c2b-b28c-5741d1beb817 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:50.558582 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a44cb-394e-4c2b-b28c-5741d1beb817 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:50.558785 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a44cb-394e-4c2b-b28c-5741d1beb817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:50.575372 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-7b5a44cb-394e-4c2b-b28c-5741d1beb817 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:50.576556 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-7b5a44cb-394e-4c2b-b28c-5741d1beb817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:50.641412 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 578/1136] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:50.667597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d2886-719e-44e1-9d7f-fd0bbbe89973 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:50.761027 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:50.764728 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d2886-719e-44e1-9d7f-fd0bbbe89973 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:50.798621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33bfa7-2c45-48f9-aa56-ea69f282097a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:50.799451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33bfa7-2c45-48f9-aa56-ea69f282097a None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:50.799728 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33bfa7-2c45-48f9-aa56-ea69f282097a None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:50.800001 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33bfa7-2c45-48f9-aa56-ea69f282097a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:50.801222 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33bfa7-2c45-48f9-aa56-ea69f282097a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:50.802589 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33bfa7-2c45-48f9-aa56-ea69f282097a None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:50.804184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 559/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:28:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:50.810076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a44cb-394e-4c2b-b28c-5741d1beb817 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:50.835109 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:50.908398 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a44cb-394e-4c2b-b28c-5741d1beb817 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:50.941580 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d5d95-3ea1-4d4c-a82c-3faf73467c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:50.942238 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d5d95-3ea1-4d4c-a82c-3faf73467c11 None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:50.942524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d5d95-3ea1-4d4c-a82c-3faf73467c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:50.942804 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d5d95-3ea1-4d4c-a82c-3faf73467c11 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:50.943505 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d5d95-3ea1-4d4c-a82c-3faf73467c11 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:50.944947 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d5d95-3ea1-4d4c-a82c-3faf73467c11 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:51.014318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 579/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 22:28:50 2022] POST /identity/v3/users => generated 286 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:28:53.054677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86766f09-8caa-41dd-9365-bd94a198ecb3 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:53.055205 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86766f09-8caa-41dd-9365-bd94a198ecb3 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:53.055324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86766f09-8caa-41dd-9365-bd94a198ecb3 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:53.056596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 560/1139] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:28:53.061770 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9e1a6-0448-4dc7-a18e-fdee672ccb10 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:53.062140 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9e1a6-0448-4dc7-a18e-fdee672ccb10 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:53.062339 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9e1a6-0448-4dc7-a18e-fdee672ccb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:53.078741 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-27f9e1a6-0448-4dc7-a18e-fdee672ccb10 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:53.079923 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-27f9e1a6-0448-4dc7-a18e-fdee672ccb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:53.145106 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 580/1140] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:53.169099 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86766f09-8caa-41dd-9365-bd94a198ecb3 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:53.268766 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86766f09-8caa-41dd-9365-bd94a198ecb3 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:53.302279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba10a5-c1e4-4a6a-b6d3-88ea9ca7cff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:53.303071 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba10a5-c1e4-4a6a-b6d3-88ea9ca7cff1 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:53.303358 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba10a5-c1e4-4a6a-b6d3-88ea9ca7cff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:53.303638 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba10a5-c1e4-4a6a-b6d3-88ea9ca7cff1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:53.313526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba10a5-c1e4-4a6a-b6d3-88ea9ca7cff1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:28:53.322023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-22ba10a5-c1e4-4a6a-b6d3-88ea9ca7cff1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:28:53.322023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:53.322023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:53.322023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:53.322023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:53.322023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:53.322023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:53.322023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:53.322023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:53.322023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:53.322023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:53.322023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:53.322023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:28:53.322023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:28:53.322023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:28:53.322023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:28:53.323187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:28:53.323187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:28:53.323187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:28:53.323187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:28:53.323187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:53.323187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:53.323187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:53.323187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:53.323187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:53.323187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:53.323187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:53.323187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:53.323187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:53.323187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:53.323187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:53.323187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:53.324356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:53.324356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:53.324356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:53.324356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:53.324356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:28:53.324356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:28:53.324356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:28:53.324356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:28:53.324356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:28:53.324356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:28:53.324356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:28:53.324356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:53.324356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 561/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:28:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:53.329386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f9e1a6-0448-4dc7-a18e-fdee672ccb10 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:53.341456 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:53.426062 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f9e1a6-0448-4dc7-a18e-fdee672ccb10 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:53.460496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b25ee-e3ea-411c-beb3-493d785c3e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:53.461175 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b25ee-e3ea-411c-beb3-493d785c3e64 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:53.461460 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b25ee-e3ea-411c-beb3-493d785c3e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:53.461737 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b25ee-e3ea-411c-beb3-493d785c3e64 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:53.462472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b25ee-e3ea-411c-beb3-493d785c3e64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:53.462777 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b25ee-e3ea-411c-beb3-493d785c3e64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:53.464120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b25ee-e3ea-411c-beb3-493d785c3e64 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:53.472009 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 581/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:28:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:53.477735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba10a5-c1e4-4a6a-b6d3-88ea9ca7cff1 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:53.506124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:53.575107 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba10a5-c1e4-4a6a-b6d3-88ea9ca7cff1 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:53.607651 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e424f0-3636-43e2-9dad-4618a7248146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:53.608504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e424f0-3636-43e2-9dad-4618a7248146 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:53.608815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e424f0-3636-43e2-9dad-4618a7248146 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:53.609094 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e424f0-3636-43e2-9dad-4618a7248146 None admin] PATH_INFO: `/v3/users/5e8d35ce153e4d68845afafcc043caf0` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:53.622986 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e424f0-3636-43e2-9dad-4618a7248146 None admin] RBAC: Authorizing `identity:get_user(user_id=5e8d35ce153e4d68845afafcc043caf0)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:53.624416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e424f0-3636-43e2-9dad-4618a7248146 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:53.626075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 562/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:28:53 2022] GET /identity/v3/users/5e8d35ce153e4d68845afafcc043caf0 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:53.632489 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b25ee-e3ea-411c-beb3-493d785c3e64 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:53.699683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:53.728939 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b25ee-e3ea-411c-beb3-493d785c3e64 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:53.762431 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72665fb2-b0c7-4f04-a417-04e396680612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:53.763064 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72665fb2-b0c7-4f04-a417-04e396680612 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:53.763350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72665fb2-b0c7-4f04-a417-04e396680612 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:53.763632 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72665fb2-b0c7-4f04-a417-04e396680612 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:53.828408 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72665fb2-b0c7-4f04-a417-04e396680612 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:53.830014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72665fb2-b0c7-4f04-a417-04e396680612 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:53.895586 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:53.925025 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 582/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:28:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:28:53.930921 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e424f0-3636-43e2-9dad-4618a7248146 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:54.013978 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:54.027782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e424f0-3636-43e2-9dad-4618a7248146 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:54.061620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c959ba-d816-4dfb-9f1e-b9198faf6d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:54.062491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c959ba-d816-4dfb-9f1e-b9198faf6d05 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:54.062774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c959ba-d816-4dfb-9f1e-b9198faf6d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:54.063054 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c959ba-d816-4dfb-9f1e-b9198faf6d05 None admin] PATH_INFO: `/v3/projects/3097a2892abd4e0caed6de369115154a` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:54.064202 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c959ba-d816-4dfb-9f1e-b9198faf6d05 None admin] RBAC: Authorizing `identity:get_project(project_id=3097a2892abd4e0caed6de369115154a)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:54.065837 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c959ba-d816-4dfb-9f1e-b9198faf6d05 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:54.067501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 563/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:28:53 2022] GET /identity/v3/projects/3097a2892abd4e0caed6de369115154a => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:54.073482 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72665fb2-b0c7-4f04-a417-04e396680612 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:54.168786 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72665fb2-b0c7-4f04-a417-04e396680612 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:54.201339 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ae8b0-f0df-4108-ab03-6b7f4f9cd2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:54.201976 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ae8b0-f0df-4108-ab03-6b7f4f9cd2ff None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:54.202259 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ae8b0-f0df-4108-ab03-6b7f4f9cd2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:54.202532 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ae8b0-f0df-4108-ab03-6b7f4f9cd2ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:54.203343 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ae8b0-f0df-4108-ab03-6b7f4f9cd2ff None admin] RBAC: Adding query filter params (role.id=6ff7708b59984b9287cc81d46a57b4b4, scope.project.id=3097a2892abd4e0caed6de369115154a, user.id=5e8d35ce153e4d68845afafcc043caf0) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:54.203652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ae8b0-f0df-4108-ab03-6b7f4f9cd2ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:54.205103 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ae8b0-f0df-4108-ab03-6b7f4f9cd2ff None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:54.210562 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 583/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:28:54 2022] GET /identity/v3/role_assignments?user.id=5e8d35ce153e4d68845afafcc043caf0&scope.project.id=3097a2892abd4e0caed6de369115154a&role.id=6ff7708b59984b9287cc81d46a57b4b4 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:56.197295 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbc4a0-343e-409b-99f7-ca319ae73c8d None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:56.197831 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbc4a0-343e-409b-99f7-ca319ae73c8d None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:56.197831 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbc4a0-343e-409b-99f7-ca319ae73c8d None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:56.199229 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 564/1147] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:56.204463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f25fd-0d72-4995-a92a-3f96a7b1fda8 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:56.204908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f25fd-0d72-4995-a92a-3f96a7b1fda8 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:56.205056 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f25fd-0d72-4995-a92a-3f96a7b1fda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:56.222665 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-ef8f25fd-0d72-4995-a92a-3f96a7b1fda8 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:56.223846 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ef8f25fd-0d72-4995-a92a-3f96a7b1fda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:56.251554 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:56.289650 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 584/1148] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:56.311972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbc4a0-343e-409b-99f7-ca319ae73c8d None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:56.410065 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbc4a0-343e-409b-99f7-ca319ae73c8d None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:56.443025 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe24f4-052c-48af-9240-5f6a0efe4015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:56.443807 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe24f4-052c-48af-9240-5f6a0efe4015 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:56.444092 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe24f4-052c-48af-9240-5f6a0efe4015 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:56.444370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe24f4-052c-48af-9240-5f6a0efe4015 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:56.454147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe24f4-052c-48af-9240-5f6a0efe4015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:28:56.462467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-2cbe24f4-052c-48af-9240-5f6a0efe4015 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:28:56.462467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:56.462467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:56.462467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:56.462467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:56.462467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:56.462467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:56.462467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:56.462467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:56.462467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:56.462467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:56.462467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:56.462467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:28:56.462467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:28:56.462467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:28:56.462467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:28:56.463555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:28:56.463555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:28:56.463555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:28:56.463555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:28:56.463555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:56.463555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:56.463555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:56.463555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:56.463555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:56.463555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:56.463555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:56.463555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:56.463555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:56.463555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:56.463555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:56.463555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:56.464436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:56.464436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:56.464436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:56.464436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:56.464436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:28:56.464436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:28:56.464436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:28:56.464436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:28:56.464436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:28:56.464436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:28:56.464436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:28:56.464436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:28:56.464436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 565/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:28:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:56.469700 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f25fd-0d72-4995-a92a-3f96a7b1fda8 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:56.565884 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f25fd-0d72-4995-a92a-3f96a7b1fda8 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:56.598704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f2bbb-2b85-4abf-a554-759f7b2e19f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:56.599364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f2bbb-2b85-4abf-a554-759f7b2e19f0 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:56.599654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f2bbb-2b85-4abf-a554-759f7b2e19f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:56.599934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f2bbb-2b85-4abf-a554-759f7b2e19f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:56.600858 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f2bbb-2b85-4abf-a554-759f7b2e19f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:56.601175 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f2bbb-2b85-4abf-a554-759f7b2e19f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:56.602540 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f2bbb-2b85-4abf-a554-759f7b2e19f0 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:56.610305 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 585/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:28:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:56.616633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe24f4-052c-48af-9240-5f6a0efe4015 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:56.632951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:56.714490 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe24f4-052c-48af-9240-5f6a0efe4015 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:56.746906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa36c3-563c-48f3-a328-14c6cfc4bafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:56.747752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa36c3-563c-48f3-a328-14c6cfc4bafb None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:56.748044 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa36c3-563c-48f3-a328-14c6cfc4bafb None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:56.748324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa36c3-563c-48f3-a328-14c6cfc4bafb None admin] PATH_INFO: `/v3/users/5e8d35ce153e4d68845afafcc043caf0` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:56.749523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa36c3-563c-48f3-a328-14c6cfc4bafb None admin] RBAC: Authorizing `identity:get_user(user_id=5e8d35ce153e4d68845afafcc043caf0)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:56.750943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa36c3-563c-48f3-a328-14c6cfc4bafb None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:56.753321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 566/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:28:56 2022] GET /identity/v3/users/5e8d35ce153e4d68845afafcc043caf0 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:56.758926 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f2bbb-2b85-4abf-a554-759f7b2e19f0 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:56.787160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:56.856210 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f2bbb-2b85-4abf-a554-759f7b2e19f0 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:56.888972 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b798d35-5c82-48c3-b8b1-920149d47b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:56.889818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b798d35-5c82-48c3-b8b1-920149d47b00 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:56.890104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b798d35-5c82-48c3-b8b1-920149d47b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:56.890383 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b798d35-5c82-48c3-b8b1-920149d47b00 None admin] PATH_INFO: `/v3/projects/3097a2892abd4e0caed6de369115154a` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:56.891551 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b798d35-5c82-48c3-b8b1-920149d47b00 None admin] RBAC: Authorizing `identity:get_project(project_id=3097a2892abd4e0caed6de369115154a)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:56.893231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b798d35-5c82-48c3-b8b1-920149d47b00 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:56.894900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 586/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:28:56 2022] GET /identity/v3/projects/3097a2892abd4e0caed6de369115154a => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:56.900779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa36c3-563c-48f3-a328-14c6cfc4bafb None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:56.979441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:56.996990 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa36c3-563c-48f3-a328-14c6cfc4bafb None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:57.029454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efe733-634b-4a6e-9b2e-a9f3fed81079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:57.030332 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efe733-634b-4a6e-9b2e-a9f3fed81079 None admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:57.030623 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efe733-634b-4a6e-9b2e-a9f3fed81079 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:57.030903 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efe733-634b-4a6e-9b2e-a9f3fed81079 None admin] PATH_INFO: `/v3/projects/3097a2892abd4e0caed6de369115154a/users/5e8d35ce153e4d68845afafcc043caf0/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:57.032894 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efe733-634b-4a6e-9b2e-a9f3fed81079 None admin] RBAC: Authorizing `identity:create_grant(project_id=3097a2892abd4e0caed6de369115154a, user_id=5e8d35ce153e4d68845afafcc043caf0, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:57.034900 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efe733-634b-4a6e-9b2e-a9f3fed81079 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:57.104825 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 567/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:28:56 2022] PUT /identity/v3/projects/3097a2892abd4e0caed6de369115154a/users/5e8d35ce153e4d68845afafcc043caf0/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:28:59.134194 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fd9f5-4660-4b9a-95b6-82d67e662596 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:59.134727 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fd9f5-4660-4b9a-95b6-82d67e662596 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:59.134847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fd9f5-4660-4b9a-95b6-82d67e662596 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:59.136126 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 587/1154] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:59.141231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d09bd-52ce-458e-8bac-587087c242b4 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:59.141648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d09bd-52ce-458e-8bac-587087c242b4 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:59.141788 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d09bd-52ce-458e-8bac-587087c242b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:59.159172 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-5a0d09bd-52ce-458e-8bac-587087c242b4 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:59.160429 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-5a0d09bd-52ce-458e-8bac-587087c242b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:59.224148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 568/1155] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:28:59.247580 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845fd9f5-4660-4b9a-95b6-82d67e662596 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:59.341952 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:59.348628 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845fd9f5-4660-4b9a-95b6-82d67e662596 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:59.381753 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5f5ec-df19-4b3f-9081-b9d39a9896d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:59.382533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5f5ec-df19-4b3f-9081-b9d39a9896d6 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:59.382819 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5f5ec-df19-4b3f-9081-b9d39a9896d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:59.383096 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5f5ec-df19-4b3f-9081-b9d39a9896d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:59.393618 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5f5ec-df19-4b3f-9081-b9d39a9896d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:28:59.401201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-8ec5f5ec-df19-4b3f-9081-b9d39a9896d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:28:59.401201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:28:59.401201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:28:59.401201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:28:59.401201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:28:59.401201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:28:59.401201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:28:59.401201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:28:59.401201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:28:59.401201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:28:59.401201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:28:59.401201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:28:59.401201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:28:59.401201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:28:59.401201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:28:59.401201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:28:59.402265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:28:59.402265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:28:59.402265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:28:59.402265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:28:59.402265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:28:59.402265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:28:59.402265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:28:59.402265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:28:59.402265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:28:59.402265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:28:59.402265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:28:59.402265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:28:59.402265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:28:59.402265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:28:59.402265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:28:59.402265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:28:59.403313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:28:59.403313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:28:59.403313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:28:59.403313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:28:59.403313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:28:59.403313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:28:59.403313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:28:59.403313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:28:59.403313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:28:59.403313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:28:59.403313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:28:59.403313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:28:59.403313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 588/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:28:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:28:59.408906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d09bd-52ce-458e-8bac-587087c242b4 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:59.476965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:59.505059 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d09bd-52ce-458e-8bac-587087c242b4 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:59.538888 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292617d-1899-442a-92f8-2cb4df33d451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:59.539542 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292617d-1899-442a-92f8-2cb4df33d451 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:59.539825 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292617d-1899-442a-92f8-2cb4df33d451 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:59.540099 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292617d-1899-442a-92f8-2cb4df33d451 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:59.540866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292617d-1899-442a-92f8-2cb4df33d451 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:28:59.541173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292617d-1899-442a-92f8-2cb4df33d451 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:59.542527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292617d-1899-442a-92f8-2cb4df33d451 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:59.550530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 569/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:28:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:59.556190 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5f5ec-df19-4b3f-9081-b9d39a9896d6 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:59.653566 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5f5ec-df19-4b3f-9081-b9d39a9896d6 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:59.685921 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd6b88-a665-4cac-bf8b-fd3456cf41c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:59.686777 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd6b88-a665-4cac-bf8b-fd3456cf41c5 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:59.687064 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd6b88-a665-4cac-bf8b-fd3456cf41c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:59.687344 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd6b88-a665-4cac-bf8b-fd3456cf41c5 None admin] PATH_INFO: `/v3/users/5e8d35ce153e4d68845afafcc043caf0` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:59.688518 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd6b88-a665-4cac-bf8b-fd3456cf41c5 None admin] RBAC: Authorizing `identity:get_user(user_id=5e8d35ce153e4d68845afafcc043caf0)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:59.690562 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd6b88-a665-4cac-bf8b-fd3456cf41c5 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:59.692173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 589/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:28:59 2022] GET /identity/v3/users/5e8d35ce153e4d68845afafcc043caf0 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:28:59.697862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292617d-1899-442a-92f8-2cb4df33d451 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:59.797302 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292617d-1899-442a-92f8-2cb4df33d451 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:59.829889 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70f4c2-815c-4339-93c8-1493727ffa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:59.830663 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70f4c2-815c-4339-93c8-1493727ffa55 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:59.830942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70f4c2-815c-4339-93c8-1493727ffa55 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:59.831217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70f4c2-815c-4339-93c8-1493727ffa55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:59.833849 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:59.895984 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70f4c2-815c-4339-93c8-1493727ffa55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:59.897443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70f4c2-815c-4339-93c8-1493727ffa55 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:59.990604 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 570/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:28:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:28:59.996400 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd6b88-a665-4cac-bf8b-fd3456cf41c5 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:00.020840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:00.093115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd6b88-a665-4cac-bf8b-fd3456cf41c5 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:00.126054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bcd0f0-5cfc-4413-abea-105ae414fc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:00.126958 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcd0f0-5cfc-4413-abea-105ae414fc01 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:00.127304 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcd0f0-5cfc-4413-abea-105ae414fc01 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:00.127576 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcd0f0-5cfc-4413-abea-105ae414fc01 None admin] PATH_INFO: `/v3/projects/3097a2892abd4e0caed6de369115154a` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:00.128773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bcd0f0-5cfc-4413-abea-105ae414fc01 None admin] RBAC: Authorizing `identity:get_project(project_id=3097a2892abd4e0caed6de369115154a)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:00.130197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bcd0f0-5cfc-4413-abea-105ae414fc01 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:00.132003 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 590/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:28:59 2022] GET /identity/v3/projects/3097a2892abd4e0caed6de369115154a => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:29:00.137933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70f4c2-815c-4339-93c8-1493727ffa55 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:00.170791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:00.233799 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70f4c2-815c-4339-93c8-1493727ffa55 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:00.266710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ae206-7e15-496f-a7d2-d5424a73f878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:00.267301 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ae206-7e15-496f-a7d2-d5424a73f878 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:00.267796 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ae206-7e15-496f-a7d2-d5424a73f878 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:00.268083 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ae206-7e15-496f-a7d2-d5424a73f878 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:00.268947 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ae206-7e15-496f-a7d2-d5424a73f878 None admin] RBAC: Adding query filter params (role.id=6ff7708b59984b9287cc81d46a57b4b4, scope.project.id=3097a2892abd4e0caed6de369115154a, user.id=5e8d35ce153e4d68845afafcc043caf0) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:29:00.269262 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ae206-7e15-496f-a7d2-d5424a73f878 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:00.270707 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ae206-7e15-496f-a7d2-d5424a73f878 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:00.276623 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 571/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:29:00 2022] GET /identity/v3/role_assignments?user.id=5e8d35ce153e4d68845afafcc043caf0&scope.project.id=3097a2892abd4e0caed6de369115154a&role.id=6ff7708b59984b9287cc81d46a57b4b4 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:29:02.303340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac5ed7-cb25-4ac3-8f09-8f12c40c196c None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:02.303778 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac5ed7-cb25-4ac3-8f09-8f12c40c196c None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:02.303899 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac5ed7-cb25-4ac3-8f09-8f12c40c196c None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:02.305287 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 591/1162] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:02.310333 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773b933-1725-4ecf-a4d9-7e4419d90989 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:02.310725 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773b933-1725-4ecf-a4d9-7e4419d90989 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:02.310908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773b933-1725-4ecf-a4d9-7e4419d90989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:02.327662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-3773b933-1725-4ecf-a4d9-7e4419d90989 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:02.328869 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3773b933-1725-4ecf-a4d9-7e4419d90989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:02.393579 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 572/1163] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:29:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:29:02.416136 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac5ed7-cb25-4ac3-8f09-8f12c40c196c None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:02.505227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:02.514357 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac5ed7-cb25-4ac3-8f09-8f12c40c196c None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:02.547378 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0b7b8-8b8a-477f-adbb-b7c5611b8850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:02.547955 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0b7b8-8b8a-477f-adbb-b7c5611b8850 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:02.548239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0b7b8-8b8a-477f-adbb-b7c5611b8850 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:02.548536 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0b7b8-8b8a-477f-adbb-b7c5611b8850 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:02.614087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0b7b8-8b8a-477f-adbb-b7c5611b8850 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:02.615522 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0b7b8-8b8a-477f-adbb-b7c5611b8850 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:02.700747 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:02.709619 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 592/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:29:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:29:02.715585 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773b933-1725-4ecf-a4d9-7e4419d90989 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:02.742299 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:02.810691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773b933-1725-4ecf-a4d9-7e4419d90989 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:02.843473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f347a-0635-4afa-9c05-0bbddc26acd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:02.844263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f347a-0635-4afa-9c05-0bbddc26acd4 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:02.844545 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f347a-0635-4afa-9c05-0bbddc26acd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:02.844838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f347a-0635-4afa-9c05-0bbddc26acd4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:02.851324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f347a-0635-4afa-9c05-0bbddc26acd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:02.852781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f347a-0635-4afa-9c05-0bbddc26acd4 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:02.867177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-389f347a-0635-4afa-9c05-0bbddc26acd4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 22:29:02.867177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:29:02.867177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 23 22:29:02.867177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 23 22:29:02.867177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:29:02.867177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:29:02.867177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:29:02.867177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:29:02.867177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 23 22:29:02.867177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:29:02.867177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:29:02.867177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:29:02.867177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:29:02.867177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:29:02.867177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:29:02.867177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:29:02.867177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:29:02.868390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:29:02.868390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:29:02.868390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:29:02.868390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:29:02.868390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:29:02.868390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:29:02.868390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:29:02.868390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:29:02.868390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:29:02.868390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:29:02.868390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:29:02.868390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:29:02.868390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:29:02.868390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:29:02.868390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:29:02.868390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:29:02.869637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:29:02.869637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:29:02.869637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:29:02.869637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:29:02.869637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:29:02.869637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:29:02.869637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:29:02.869637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:29:02.869637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:29:02.869637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:29:02.869637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 23 22:29:02.869637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:29:02.869637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 22:29:02.869637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:29:02.869637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 573/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 23 22:29:02 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:29:02.874167 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0b7b8-8b8a-477f-adbb-b7c5611b8850 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:02.970041 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0b7b8-8b8a-477f-adbb-b7c5611b8850 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:03.002702 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813097c-a87d-4d22-bab9-885a61f792d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:03.003367 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813097c-a87d-4d22-bab9-885a61f792d0 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:03.003650 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813097c-a87d-4d22-bab9-885a61f792d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:03.003924 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813097c-a87d-4d22-bab9-885a61f792d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:03.004662 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813097c-a87d-4d22-bab9-885a61f792d0 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:29:03.004970 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813097c-a87d-4d22-bab9-885a61f792d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:03.006315 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813097c-a87d-4d22-bab9-885a61f792d0 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:03.013191 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 593/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 23 22:29:02 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:29:05.065074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b80343-bee0-4340-9a01-90cadc53fd01 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:05.065606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b80343-bee0-4340-9a01-90cadc53fd01 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:05.065606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b80343-bee0-4340-9a01-90cadc53fd01 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:05.066993 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 574/1167] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:05.072255 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430dec7-5da7-4754-87d4-5b3e3b81cdae None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:05.072673 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430dec7-5da7-4754-87d4-5b3e3b81cdae None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:05.072870 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430dec7-5da7-4754-87d4-5b3e3b81cdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:05.090296 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-6430dec7-5da7-4754-87d4-5b3e3b81cdae None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:05.091487 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6430dec7-5da7-4754-87d4-5b3e3b81cdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:05.141318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:05.157593 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 594/1168] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:29:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:29:05.181638 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b80343-bee0-4340-9a01-90cadc53fd01 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:05.271225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:05.283887 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b80343-bee0-4340-9a01-90cadc53fd01 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:05.317610 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362d11c-281d-4f95-bdb8-947e60ada213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:05.318216 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362d11c-281d-4f95-bdb8-947e60ada213 None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:05.318503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362d11c-281d-4f95-bdb8-947e60ada213 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:05.318785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362d11c-281d-4f95-bdb8-947e60ada213 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:05.319535 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362d11c-281d-4f95-bdb8-947e60ada213 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:05.320923 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362d11c-281d-4f95-bdb8-947e60ada213 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:05.409669 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 575/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Thu Jun 23 22:29:05 2022] POST /identity/v3/domains => generated 267 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:29:07.455130 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00e98e-8bed-48db-92a9-2c64986fd706 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:07.455688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00e98e-8bed-48db-92a9-2c64986fd706 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:07.455793 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00e98e-8bed-48db-92a9-2c64986fd706 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:07.457149 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 595/1170] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:29:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:07.462308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d217c-c95a-4e57-9e64-4b7194bd2c6f None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:07.462711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d217c-c95a-4e57-9e64-4b7194bd2c6f None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:07.462875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d217c-c95a-4e57-9e64-4b7194bd2c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:07.479916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-bc5d217c-c95a-4e57-9e64-4b7194bd2c6f None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:07.481152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-bc5d217c-c95a-4e57-9e64-4b7194bd2c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:07.545837 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 576/1171] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:29:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:29:07.569350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00e98e-8bed-48db-92a9-2c64986fd706 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:07.667676 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00e98e-8bed-48db-92a9-2c64986fd706 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:07.700716 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca6c2e-4a01-4a3a-9f75-0ab76e6f5b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:07.701563 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca6c2e-4a01-4a3a-9f75-0ab76e6f5b10 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:07.701852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca6c2e-4a01-4a3a-9f75-0ab76e6f5b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:07.702131 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca6c2e-4a01-4a3a-9f75-0ab76e6f5b10 None admin] PATH_INFO: `/v3/domains/77e25c75940140f2b0dc4a13aaf39c8d` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:07.712977 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ca6c2e-4a01-4a3a-9f75-0ab76e6f5b10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=77e25c75940140f2b0dc4a13aaf39c8d)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:07.714420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ca6c2e-4a01-4a3a-9f75-0ab76e6f5b10 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:07.716049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 596/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Thu Jun 23 22:29:07 2022] GET /identity/v3/domains/77e25c75940140f2b0dc4a13aaf39c8d => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:29:07.721840 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d217c-c95a-4e57-9e64-4b7194bd2c6f None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:07.795739 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:07.816067 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d217c-c95a-4e57-9e64-4b7194bd2c6f None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:07.848532 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff49f2-8843-4771-80ca-fa2bc9beefe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:07.849203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff49f2-8843-4771-80ca-fa2bc9beefe1 None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:07.849487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff49f2-8843-4771-80ca-fa2bc9beefe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:07.849766 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff49f2-8843-4771-80ca-fa2bc9beefe1 None admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:07.850456 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff49f2-8843-4771-80ca-fa2bc9beefe1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:07.851873 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff49f2-8843-4771-80ca-fa2bc9beefe1 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:07.924051 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 577/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 22:29:07 2022] POST /identity/v3/projects => generated 361 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:29:09.944678 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66292836-202c-4e0b-b111-029bed13871c None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:09.945245 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66292836-202c-4e0b-b111-029bed13871c None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:09.945245 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66292836-202c-4e0b-b111-029bed13871c None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:09.946616 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 597/1174] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:09.952095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f188e08-c288-4c92-9f01-15fb94ca8354 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:09.952465 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f188e08-c288-4c92-9f01-15fb94ca8354 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:09.952693 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f188e08-c288-4c92-9f01-15fb94ca8354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:09.970033 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-0f188e08-c288-4c92-9f01-15fb94ca8354 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:09.971386 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-0f188e08-c288-4c92-9f01-15fb94ca8354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:10.035973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 578/1175] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:29:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:29:10.061165 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66292836-202c-4e0b-b111-029bed13871c None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:10.107914 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:10.161556 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66292836-202c-4e0b-b111-029bed13871c None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:10.194761 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257c89e-7525-46ab-ae07-e515dfd5e2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:10.195548 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257c89e-7525-46ab-ae07-e515dfd5e2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:10.195830 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257c89e-7525-46ab-ae07-e515dfd5e2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:10.196108 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257c89e-7525-46ab-ae07-e515dfd5e2e1 None admin] PATH_INFO: `/v3/domains/77e25c75940140f2b0dc4a13aaf39c8d` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:10.197253 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257c89e-7525-46ab-ae07-e515dfd5e2e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=77e25c75940140f2b0dc4a13aaf39c8d)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:10.198672 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257c89e-7525-46ab-ae07-e515dfd5e2e1 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:10.200267 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 598/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Thu Jun 23 22:29:10 2022] GET /identity/v3/domains/77e25c75940140f2b0dc4a13aaf39c8d => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:29:10.206018 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f188e08-c288-4c92-9f01-15fb94ca8354 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:10.263167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:10.304548 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f188e08-c288-4c92-9f01-15fb94ca8354 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:10.337655 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6515b8-c349-4907-aed2-bfd59eb5c167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:10.338317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6515b8-c349-4907-aed2-bfd59eb5c167 None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:10.338599 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6515b8-c349-4907-aed2-bfd59eb5c167 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:10.338872 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6515b8-c349-4907-aed2-bfd59eb5c167 None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:10.339582 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6515b8-c349-4907-aed2-bfd59eb5c167 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:10.341045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6515b8-c349-4907-aed2-bfd59eb5c167 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:10.396322 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 579/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 22:29:10 2022] POST /identity/v3/users => generated 311 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:29:12.430021 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0b5e9-d1a7-478d-bfef-4b6cd8570061 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:12.430533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0b5e9-d1a7-478d-bfef-4b6cd8570061 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:12.430619 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0b5e9-d1a7-478d-bfef-4b6cd8570061 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:12.431927 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 599/1178] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:12.437121 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb68e76-b59c-41dc-a461-d4e96bb28086 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:12.437518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb68e76-b59c-41dc-a461-d4e96bb28086 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:12.437685 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb68e76-b59c-41dc-a461-d4e96bb28086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:12.455547 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-eeb68e76-b59c-41dc-a461-d4e96bb28086 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:12.456760 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-eeb68e76-b59c-41dc-a461-d4e96bb28086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:12.521283 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 580/1179] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:29:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:29:12.545160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0b5e9-d1a7-478d-bfef-4b6cd8570061 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:12.648856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0b5e9-d1a7-478d-bfef-4b6cd8570061 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:12.662463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:12.681542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c5e57-97e7-47f5-a9f9-0680950a543a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:12.682326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c5e57-97e7-47f5-a9f9-0680950a543a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:12.682609 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c5e57-97e7-47f5-a9f9-0680950a543a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:12.682882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c5e57-97e7-47f5-a9f9-0680950a543a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:12.693801 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c5e57-97e7-47f5-a9f9-0680950a543a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:29:12.701560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-413c5e57-97e7-47f5-a9f9-0680950a543a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:29:12.701560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:29:12.701560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:29:12.701560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:29:12.701560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:29:12.701560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:29:12.701560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:29:12.701560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:29:12.701560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:29:12.701560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:29:12.701560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:29:12.701560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:29:12.701560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:29:12.701560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:29:12.701560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:29:12.701560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:29:12.702688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:29:12.702688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:29:12.702688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:29:12.702688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:29:12.702688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:29:12.702688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:29:12.702688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:29:12.702688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:29:12.702688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:29:12.702688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:29:12.702688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:29:12.702688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:29:12.702688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:29:12.702688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:29:12.702688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:29:12.702688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:29:12.703809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:29:12.703809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:29:12.703809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:29:12.703809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:29:12.703809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:29:12.703809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:29:12.703809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:29:12.703809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:29:12.703809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:29:12.703809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:29:12.703809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:29:12.703809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:29:12.703809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 600/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:29:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:29:12.709216 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb68e76-b59c-41dc-a461-d4e96bb28086 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:12.754335 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:12.806281 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb68e76-b59c-41dc-a461-d4e96bb28086 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:12.839444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5a488-1e27-4e22-9904-341ab86beb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:12.840101 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5a488-1e27-4e22-9904-341ab86beb06 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:12.840386 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5a488-1e27-4e22-9904-341ab86beb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:12.840696 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5a488-1e27-4e22-9904-341ab86beb06 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:12.841435 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5a488-1e27-4e22-9904-341ab86beb06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:29:12.841744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5a488-1e27-4e22-9904-341ab86beb06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:12.843170 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5a488-1e27-4e22-9904-341ab86beb06 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:12.851751 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 581/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:29:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:29:12.857312 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c5e57-97e7-47f5-a9f9-0680950a543a None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:12.953163 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c5e57-97e7-47f5-a9f9-0680950a543a None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:12.985856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1d19c-9f8e-48af-96ae-882ba7775ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:12.986709 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1d19c-9f8e-48af-96ae-882ba7775ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:12.986996 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1d19c-9f8e-48af-96ae-882ba7775ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:12.987272 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1d19c-9f8e-48af-96ae-882ba7775ea4 None admin] PATH_INFO: `/v3/users/dd4fc4aa21b244cc83c50c45ebf4e4a1` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:12.999755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1d19c-9f8e-48af-96ae-882ba7775ea4 None admin] RBAC: Authorizing `identity:get_user(user_id=dd4fc4aa21b244cc83c50c45ebf4e4a1)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:13.001755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1d19c-9f8e-48af-96ae-882ba7775ea4 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:13.003373 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 601/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:29:12 2022] GET /identity/v3/users/dd4fc4aa21b244cc83c50c45ebf4e4a1 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:29:13.008962 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5a488-1e27-4e22-9904-341ab86beb06 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:13.104768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5a488-1e27-4e22-9904-341ab86beb06 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:13.114258 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:13.138693 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47b7ce-0922-407a-a21d-737a784cc1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:13.139330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47b7ce-0922-407a-a21d-737a784cc1ac None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:13.139618 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47b7ce-0922-407a-a21d-737a784cc1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:13.139897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47b7ce-0922-407a-a21d-737a784cc1ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:13.205548 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47b7ce-0922-407a-a21d-737a784cc1ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:13.206983 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47b7ce-0922-407a-a21d-737a784cc1ac None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:13.306944 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 582/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:29:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:29:13.312755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1d19c-9f8e-48af-96ae-882ba7775ea4 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:13.344412 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:13.411044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1d19c-9f8e-48af-96ae-882ba7775ea4 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:13.506776 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5281c3-f7ae-48a6-9c4d-27f07765917a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:13.507636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5281c3-f7ae-48a6-9c4d-27f07765917a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:13.507920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5281c3-f7ae-48a6-9c4d-27f07765917a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:13.508192 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5281c3-f7ae-48a6-9c4d-27f07765917a None admin] PATH_INFO: `/v3/projects/f141174a089c49918a4b7e7b2b53ef6c` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:13.509395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5281c3-f7ae-48a6-9c4d-27f07765917a None admin] RBAC: Authorizing `identity:get_project(project_id=f141174a089c49918a4b7e7b2b53ef6c)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:13.510805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5281c3-f7ae-48a6-9c4d-27f07765917a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:13.512481 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 602/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:29:13 2022] GET /identity/v3/projects/f141174a089c49918a4b7e7b2b53ef6c => generated 361 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:29:13.519144 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47b7ce-0922-407a-a21d-737a784cc1ac None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:13.529203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:13.615369 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47b7ce-0922-407a-a21d-737a784cc1ac None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:13.647520 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bc62f-a0e7-4281-b899-c619b4539cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:13.648164 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bc62f-a0e7-4281-b899-c619b4539cdf None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:13.648450 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bc62f-a0e7-4281-b899-c619b4539cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:13.648757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bc62f-a0e7-4281-b899-c619b4539cdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:13.649579 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bc62f-a0e7-4281-b899-c619b4539cdf None admin] RBAC: Adding query filter params (role.id=6ff7708b59984b9287cc81d46a57b4b4, scope.project.id=f141174a089c49918a4b7e7b2b53ef6c, user.id=dd4fc4aa21b244cc83c50c45ebf4e4a1) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:29:13.649890 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bc62f-a0e7-4281-b899-c619b4539cdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:13.651349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bc62f-a0e7-4281-b899-c619b4539cdf None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:13.656793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 583/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:29:13 2022] GET /identity/v3/role_assignments?user.id=dd4fc4aa21b244cc83c50c45ebf4e4a1&scope.project.id=f141174a089c49918a4b7e7b2b53ef6c&role.id=6ff7708b59984b9287cc81d46a57b4b4 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:29:15.628113 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7bbf5-b79d-4c1d-a0a6-f3cb82004eb9 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:15.628685 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7bbf5-b79d-4c1d-a0a6-f3cb82004eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:15.628685 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7bbf5-b79d-4c1d-a0a6-f3cb82004eb9 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:15.629962 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 603/1186] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:15.635067 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906d93b-262d-40f0-959c-044ef0d56bfd None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:15.635457 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906d93b-262d-40f0-959c-044ef0d56bfd None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:15.635638 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906d93b-262d-40f0-959c-044ef0d56bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:15.653995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-8906d93b-262d-40f0-959c-044ef0d56bfd None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:15.655197 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-8906d93b-262d-40f0-959c-044ef0d56bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:15.720437 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 584/1187] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:29:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:29:15.742799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7bbf5-b79d-4c1d-a0a6-f3cb82004eb9 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:15.838519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:15.842246 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7bbf5-b79d-4c1d-a0a6-f3cb82004eb9 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:15.874856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d9a0b-8f99-41e4-b4f0-d9757d55045d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:15.875636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d9a0b-8f99-41e4-b4f0-d9757d55045d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:15.876444 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d9a0b-8f99-41e4-b4f0-d9757d55045d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:15.876755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d9a0b-8f99-41e4-b4f0-d9757d55045d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:15.886730 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d9a0b-8f99-41e4-b4f0-d9757d55045d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:29:15.895030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-a26d9a0b-8f99-41e4-b4f0-d9757d55045d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:29:15.895030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:29:15.895030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:29:15.895030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:29:15.895030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:29:15.895030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:29:15.895030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:29:15.895030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:29:15.895030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:29:15.895030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:29:15.895030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:29:15.895030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:29:15.895030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:29:15.895030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:29:15.895030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:29:15.895030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:29:15.896177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:29:15.896177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:29:15.896177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:29:15.896177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:29:15.896177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:29:15.896177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:29:15.896177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:29:15.896177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:29:15.896177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:29:15.896177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:29:15.896177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:29:15.896177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:29:15.896177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:29:15.896177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:29:15.896177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:29:15.896177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:29:15.897317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:29:15.897317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:29:15.897317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:29:15.897317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:29:15.897317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:29:15.897317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:29:15.897317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:29:15.897317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:29:15.897317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:29:15.897317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:29:15.897317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:29:15.897317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:29:15.897317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 604/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:29:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:29:15.902432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906d93b-262d-40f0-959c-044ef0d56bfd None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:15.904603 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:16.000277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906d93b-262d-40f0-959c-044ef0d56bfd None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:16.033543 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fcb20-632d-41c3-8034-a3dfae75dfed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:16.034196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fcb20-632d-41c3-8034-a3dfae75dfed None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:16.034483 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fcb20-632d-41c3-8034-a3dfae75dfed None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:16.034759 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fcb20-632d-41c3-8034-a3dfae75dfed None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:16.035494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024fcb20-632d-41c3-8034-a3dfae75dfed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:29:16.035804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024fcb20-632d-41c3-8034-a3dfae75dfed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:16.037203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024fcb20-632d-41c3-8034-a3dfae75dfed None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:16.045202 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 585/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:29:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:29:16.050960 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d9a0b-8f99-41e4-b4f0-d9757d55045d None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:16.147218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d9a0b-8f99-41e4-b4f0-d9757d55045d None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:16.179385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e8cd3-96fe-4acc-a0f7-abc03464a8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:16.180238 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e8cd3-96fe-4acc-a0f7-abc03464a8dc None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:16.180525 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e8cd3-96fe-4acc-a0f7-abc03464a8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:16.180831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e8cd3-96fe-4acc-a0f7-abc03464a8dc None admin] PATH_INFO: `/v3/users/dd4fc4aa21b244cc83c50c45ebf4e4a1` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:16.182009 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e8cd3-96fe-4acc-a0f7-abc03464a8dc None admin] RBAC: Authorizing `identity:get_user(user_id=dd4fc4aa21b244cc83c50c45ebf4e4a1)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:16.183424 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e8cd3-96fe-4acc-a0f7-abc03464a8dc None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:16.185050 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 605/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:29:16 2022] GET /identity/v3/users/dd4fc4aa21b244cc83c50c45ebf4e4a1 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:29:16.191263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fcb20-632d-41c3-8034-a3dfae75dfed None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:16.250561 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:16.288139 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fcb20-632d-41c3-8034-a3dfae75dfed None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:16.320820 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8871207-ad3a-4b94-b8a3-3509a33fe06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:16.321666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8871207-ad3a-4b94-b8a3-3509a33fe06b None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:16.321949 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8871207-ad3a-4b94-b8a3-3509a33fe06b None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:16.322225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8871207-ad3a-4b94-b8a3-3509a33fe06b None admin] PATH_INFO: `/v3/projects/f141174a089c49918a4b7e7b2b53ef6c` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:16.323372 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8871207-ad3a-4b94-b8a3-3509a33fe06b None admin] RBAC: Authorizing `identity:get_project(project_id=f141174a089c49918a4b7e7b2b53ef6c)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:16.324825 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8871207-ad3a-4b94-b8a3-3509a33fe06b None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:16.326504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 586/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:29:16 2022] GET /identity/v3/projects/f141174a089c49918a4b7e7b2b53ef6c => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:29:16.332598 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e8cd3-96fe-4acc-a0f7-abc03464a8dc None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:16.361317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:16.430623 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e8cd3-96fe-4acc-a0f7-abc03464a8dc None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:16.463371 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e28e9-9a8c-40e7-b5d2-b2d984ed45b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:16.464251 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e28e9-9a8c-40e7-b5d2-b2d984ed45b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:16.464539 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e28e9-9a8c-40e7-b5d2-b2d984ed45b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:16.464846 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e28e9-9a8c-40e7-b5d2-b2d984ed45b4 None admin] PATH_INFO: `/v3/projects/f141174a089c49918a4b7e7b2b53ef6c/users/dd4fc4aa21b244cc83c50c45ebf4e4a1/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:16.466801 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9e28e9-9a8c-40e7-b5d2-b2d984ed45b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f141174a089c49918a4b7e7b2b53ef6c, user_id=dd4fc4aa21b244cc83c50c45ebf4e4a1, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:16.468225 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9e28e9-9a8c-40e7-b5d2-b2d984ed45b4 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:16.585024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 606/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:29:16 2022] PUT /identity/v3/projects/f141174a089c49918a4b7e7b2b53ef6c/users/dd4fc4aa21b244cc83c50c45ebf4e4a1/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:29:18.580221 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ce48e-afbe-43f8-8df4-8df9dd22620d None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:18.580759 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ce48e-afbe-43f8-8df4-8df9dd22620d None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:18.580896 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ce48e-afbe-43f8-8df4-8df9dd22620d None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:18.582277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 587/1193] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:29:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:18.587670 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a43a79-c15a-449e-8cfd-ad08b9ee5716 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:18.588105 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a43a79-c15a-449e-8cfd-ad08b9ee5716 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:18.588217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a43a79-c15a-449e-8cfd-ad08b9ee5716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:18.605738 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-c1a43a79-c15a-449e-8cfd-ad08b9ee5716 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:18.606921 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-c1a43a79-c15a-449e-8cfd-ad08b9ee5716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:18.671708 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 607/1194] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:29:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:29:18.695077 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ce48e-afbe-43f8-8df4-8df9dd22620d None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:18.796112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ce48e-afbe-43f8-8df4-8df9dd22620d None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:18.824536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:18.829888 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ddbe6-84a4-4b9c-9143-777fec078384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:18.830733 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ddbe6-84a4-4b9c-9143-777fec078384 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:18.830973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ddbe6-84a4-4b9c-9143-777fec078384 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:18.831252 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ddbe6-84a4-4b9c-9143-777fec078384 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:18.841082 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ddbe6-84a4-4b9c-9143-777fec078384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:29:18.848795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-148ddbe6-84a4-4b9c-9143-777fec078384 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:29:18.848795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:29:18.848795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:29:18.848795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:29:18.848795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:29:18.848795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:29:18.848795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:29:18.848795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:29:18.848795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:29:18.848795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:29:18.848795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:29:18.848795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:29:18.848795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:29:18.848795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:29:18.848795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:29:18.848795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:29:18.849916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:29:18.849916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:29:18.849916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:29:18.849916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:29:18.849916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:29:18.849916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:29:18.849916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:29:18.849916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:29:18.849916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:29:18.849916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:29:18.849916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:29:18.849916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:29:18.849916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:29:18.849916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:29:18.849916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:29:18.849916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:29:18.851003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:29:18.851003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:29:18.851003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:29:18.851003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:29:18.851003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:29:18.851003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:29:18.851003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:29:18.851003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:29:18.851003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:29:18.851003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:29:18.851003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:29:18.851003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:29:18.851003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 588/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:29:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:29:18.855983 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a43a79-c15a-449e-8cfd-ad08b9ee5716 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:18.873788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:18.953072 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a43a79-c15a-449e-8cfd-ad08b9ee5716 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:18.986511 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa94da-5c7a-4610-8fa3-9b060fe04d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:18.987167 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa94da-5c7a-4610-8fa3-9b060fe04d20 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:18.987460 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa94da-5c7a-4610-8fa3-9b060fe04d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:18.987743 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa94da-5c7a-4610-8fa3-9b060fe04d20 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:18.988487 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa94da-5c7a-4610-8fa3-9b060fe04d20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:29:18.988832 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa94da-5c7a-4610-8fa3-9b060fe04d20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:18.990208 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa94da-5c7a-4610-8fa3-9b060fe04d20 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:18.998022 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 608/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:29:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:29:19.004114 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ddbe6-84a4-4b9c-9143-777fec078384 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:19.099257 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ddbe6-84a4-4b9c-9143-777fec078384 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:19.131841 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99190459-e275-4d74-9b8c-39b1a6643e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:19.132714 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99190459-e275-4d74-9b8c-39b1a6643e53 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:19.132997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99190459-e275-4d74-9b8c-39b1a6643e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:19.133271 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99190459-e275-4d74-9b8c-39b1a6643e53 None admin] PATH_INFO: `/v3/users/dd4fc4aa21b244cc83c50c45ebf4e4a1` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:19.134426 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99190459-e275-4d74-9b8c-39b1a6643e53 None admin] RBAC: Authorizing `identity:get_user(user_id=dd4fc4aa21b244cc83c50c45ebf4e4a1)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:19.135851 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99190459-e275-4d74-9b8c-39b1a6643e53 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:19.137465 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 589/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 23 22:29:19 2022] GET /identity/v3/users/dd4fc4aa21b244cc83c50c45ebf4e4a1 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:29:19.143006 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa94da-5c7a-4610-8fa3-9b060fe04d20 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:19.221139 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:19.239061 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa94da-5c7a-4610-8fa3-9b060fe04d20 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:19.271075 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258005dc-1438-4d34-91ce-9bc1e8bdbbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:19.272107 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258005dc-1438-4d34-91ce-9bc1e8bdbbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:19.272389 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258005dc-1438-4d34-91ce-9bc1e8bdbbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:19.272697 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258005dc-1438-4d34-91ce-9bc1e8bdbbb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:19.337430 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258005dc-1438-4d34-91ce-9bc1e8bdbbb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:19.338862 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258005dc-1438-4d34-91ce-9bc1e8bdbbb3 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:19.414496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:19.433530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 609/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:29:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:29:19.439990 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99190459-e275-4d74-9b8c-39b1a6643e53 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:19.498137 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:19.537227 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99190459-e275-4d74-9b8c-39b1a6643e53 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:19.569770 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ed9b0-3766-433a-9441-307f77e222fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:19.570614 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ed9b0-3766-433a-9441-307f77e222fa None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:19.570899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ed9b0-3766-433a-9441-307f77e222fa None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:19.571176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ed9b0-3766-433a-9441-307f77e222fa None admin] PATH_INFO: `/v3/projects/f141174a089c49918a4b7e7b2b53ef6c` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:19.572338 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ed9b0-3766-433a-9441-307f77e222fa None admin] RBAC: Authorizing `identity:get_project(project_id=f141174a089c49918a4b7e7b2b53ef6c)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:19.573804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ed9b0-3766-433a-9441-307f77e222fa None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:19.575486 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 590/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 23 22:29:19 2022] GET /identity/v3/projects/f141174a089c49918a4b7e7b2b53ef6c => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:29:19.581865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258005dc-1438-4d34-91ce-9bc1e8bdbbb3 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:19.677971 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258005dc-1438-4d34-91ce-9bc1e8bdbbb3 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:19.710138 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0874f23-3177-4a33-b20f-bd6459bb4d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:19.710779 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0874f23-3177-4a33-b20f-bd6459bb4d92 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:19.711063 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0874f23-3177-4a33-b20f-bd6459bb4d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:19.711341 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0874f23-3177-4a33-b20f-bd6459bb4d92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:19.712160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0874f23-3177-4a33-b20f-bd6459bb4d92 None admin] RBAC: Adding query filter params (role.id=6ff7708b59984b9287cc81d46a57b4b4, scope.project.id=f141174a089c49918a4b7e7b2b53ef6c, user.id=dd4fc4aa21b244cc83c50c45ebf4e4a1) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:29:19.712472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0874f23-3177-4a33-b20f-bd6459bb4d92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:19.713923 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0874f23-3177-4a33-b20f-bd6459bb4d92 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:19.720094 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 610/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:29:19 2022] GET /identity/v3/role_assignments?user.id=dd4fc4aa21b244cc83c50c45ebf4e4a1&scope.project.id=f141174a089c49918a4b7e7b2b53ef6c&role.id=6ff7708b59984b9287cc81d46a57b4b4 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:29:29.565837 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=36 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:29.717023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=78 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:29.675753 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40466352-2e2c-4755-8372-07b80b521cd1 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:29.676285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40466352-2e2c-4755-8372-07b80b521cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:29.676285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40466352-2e2c-4755-8372-07b80b521cd1 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:29.677672 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 591/1201] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:30:29.683200 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc951b58-6154-4c6a-a8e1-3e2245339b70 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:29.683710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc951b58-6154-4c6a-a8e1-3e2245339b70 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:29.683810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc951b58-6154-4c6a-a8e1-3e2245339b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:29.701786 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-dc951b58-6154-4c6a-a8e1-3e2245339b70 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:29.702970 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-dc951b58-6154-4c6a-a8e1-3e2245339b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:29.775790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 611/1202] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:30:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:30:29.800746 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40466352-2e2c-4755-8372-07b80b521cd1 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:29.907949 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40466352-2e2c-4755-8372-07b80b521cd1 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:29.943230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf31061-a881-4c98-a4fd-fc0a4520f310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:29.943968 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf31061-a881-4c98-a4fd-fc0a4520f310 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:29.944257 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf31061-a881-4c98-a4fd-fc0a4520f310 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:29.944539 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf31061-a881-4c98-a4fd-fc0a4520f310 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:29.955657 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf31061-a881-4c98-a4fd-fc0a4520f310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:29.957793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf31061-a881-4c98-a4fd-fc0a4520f310 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:29.974741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-3bf31061-a881-4c98-a4fd-fc0a4520f310 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:29.974741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:29.974741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:29.974741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:29.974741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:29.974741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:29.974741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:29.974741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:29.974741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:29.974741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:29.974741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:29.974741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:29.974741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:30:29.974741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:30:29.974741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:30:29.976103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:30:29.976103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:29.976103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:29.976103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:29.976103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:29.976103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:29.976103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:29.976103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:29.976103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:29.976103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:29.976103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:29.976103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:29.976103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:29.976103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:29.976103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:29.976103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:29.976103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:29.977162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:30:29.977162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:30:29.977162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:29.977162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:30:29.977162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 592/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:29.982221 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc951b58-6154-4c6a-a8e1-3e2245339b70 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:30.086582 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc951b58-6154-4c6a-a8e1-3e2245339b70 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:30.116906 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:30.122663 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94963a8-0a71-4090-9608-724dc5b9748a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:30.123323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94963a8-0a71-4090-9608-724dc5b9748a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:30.123635 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94963a8-0a71-4090-9608-724dc5b9748a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:30.123911 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94963a8-0a71-4090-9608-724dc5b9748a None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:30.124665 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94963a8-0a71-4090-9608-724dc5b9748a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:30.128897 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94963a8-0a71-4090-9608-724dc5b9748a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:30.128897 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94963a8-0a71-4090-9608-724dc5b9748a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:30.138123 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 612/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:30:30.145085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf31061-a881-4c98-a4fd-fc0a4520f310 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:30.202609 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:30.250968 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf31061-a881-4c98-a4fd-fc0a4520f310 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:30.286620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c413ec9-2636-4f89-90e1-097a65a04f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:30.287331 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c413ec9-2636-4f89-90e1-097a65a04f35 None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:30.287621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c413ec9-2636-4f89-90e1-097a65a04f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:30.287921 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c413ec9-2636-4f89-90e1-097a65a04f35 None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:30.288760 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c413ec9-2636-4f89-90e1-097a65a04f35 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:30.290257 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c413ec9-2636-4f89-90e1-097a65a04f35 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:30.422736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 593/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 23 22:30:30 2022] POST /identity/v3/users => generated 251 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:30:32.566108 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd1216-bd43-4089-83d6-2171348f264e None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:32.566611 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd1216-bd43-4089-83d6-2171348f264e None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:32.566611 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd1216-bd43-4089-83d6-2171348f264e None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:32.568506 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 613/1206] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:30:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:30:32.576384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e83de-c130-4a6b-8154-146886a21f3a None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:32.576948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e83de-c130-4a6b-8154-146886a21f3a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:32.577415 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e83de-c130-4a6b-8154-146886a21f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:32.595659 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-8d8e83de-c130-4a6b-8154-146886a21f3a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:32.600208 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-8d8e83de-c130-4a6b-8154-146886a21f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:32.674868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 594/1207] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:30:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:30:32.699253 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd1216-bd43-4089-83d6-2171348f264e None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:32.811031 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd1216-bd43-4089-83d6-2171348f264e None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:32.846380 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb1aa1-53e0-4379-aa0a-0025546eba2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:32.847152 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb1aa1-53e0-4379-aa0a-0025546eba2e None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:32.847450 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb1aa1-53e0-4379-aa0a-0025546eba2e None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:32.847732 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb1aa1-53e0-4379-aa0a-0025546eba2e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:32.859172 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb1aa1-53e0-4379-aa0a-0025546eba2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:30:32.867110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-58fb1aa1-53e0-4379-aa0a-0025546eba2e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:30:32.867110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:32.867110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:32.867110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:32.867110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:32.867110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:32.867110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:32.867110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:32.867110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:32.867110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:32.867110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:32.867110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:32.867110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:30:32.867110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:30:32.867110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:30:32.867110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:30:32.868120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:30:32.868120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:30:32.868120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:30:32.868120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:30:32.868120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:32.868120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:32.868120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:32.868120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:32.868120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:32.868120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:32.868120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:32.868120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:32.868120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:32.868120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:32.868120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:32.868120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:32.869104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:32.869104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:32.869104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:32.869104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:32.869104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:30:32.869104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:30:32.869104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:30:32.869104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:30:32.869104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:30:32.869104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:30:32.869104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:30:32.869104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:30:32.869104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 614/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:30:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:32.874540 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e83de-c130-4a6b-8154-146886a21f3a None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:32.916267 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:32.982575 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e83de-c130-4a6b-8154-146886a21f3a None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:33.018558 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027fd3f-dc40-4c75-a9dd-2f9864ddd87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:33.019355 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027fd3f-dc40-4c75-a9dd-2f9864ddd87d None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:33.019642 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027fd3f-dc40-4c75-a9dd-2f9864ddd87d None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:33.019922 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027fd3f-dc40-4c75-a9dd-2f9864ddd87d None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:33.020720 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027fd3f-dc40-4c75-a9dd-2f9864ddd87d None admin] RBAC: Adding query filter params (name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:33.021078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027fd3f-dc40-4c75-a9dd-2f9864ddd87d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:33.022457 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027fd3f-dc40-4c75-a9dd-2f9864ddd87d None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:33.031016 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 595/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:30:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:30:33.037126 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb1aa1-53e0-4379-aa0a-0025546eba2e None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:33.090512 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:33.142285 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb1aa1-53e0-4379-aa0a-0025546eba2e None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:33.177807 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558d2c4-8f0e-49e7-b3f2-25cdf7ec9f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:33.178608 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558d2c4-8f0e-49e7-b3f2-25cdf7ec9f9d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:33.178894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558d2c4-8f0e-49e7-b3f2-25cdf7ec9f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:33.179175 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558d2c4-8f0e-49e7-b3f2-25cdf7ec9f9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:33.191405 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558d2c4-8f0e-49e7-b3f2-25cdf7ec9f9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:33.199340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558d2c4-8f0e-49e7-b3f2-25cdf7ec9f9d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:33.211166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-a558d2c4-8f0e-49e7-b3f2-25cdf7ec9f9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:33.211166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:33.211166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:33.211166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:33.211166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:33.211166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:33.211166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:33.211166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:33.211166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:33.211166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:33.211166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:33.211166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:33.211166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:30:33.211166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:30:33.211166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:30:33.212261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:30:33.212261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:33.212261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:33.212261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:33.212261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:33.212261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:33.212261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:33.212261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:33.212261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:33.212261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:33.212261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:33.212261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:33.212261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:33.212261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:33.212261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:33.212261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:33.212261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:33.213325 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:30:33.213325 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:30:33.213325 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:33.213325 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:30:33.220019 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 615/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:33.222572 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027fd3f-dc40-4c75-a9dd-2f9864ddd87d None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:33.327103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:33.330466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027fd3f-dc40-4c75-a9dd-2f9864ddd87d None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:33.366082 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb42c10-1490-46ad-9c87-0a3c72db43ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:33.366751 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb42c10-1490-46ad-9c87-0a3c72db43ad None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:33.367037 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb42c10-1490-46ad-9c87-0a3c72db43ad None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:33.367319 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb42c10-1490-46ad-9c87-0a3c72db43ad None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:33.368071 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb42c10-1490-46ad-9c87-0a3c72db43ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:33.368374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb42c10-1490-46ad-9c87-0a3c72db43ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:33.369781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb42c10-1490-46ad-9c87-0a3c72db43ad None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:33.382289 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 596/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:33.387991 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558d2c4-8f0e-49e7-b3f2-25cdf7ec9f9d None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:33.493510 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558d2c4-8f0e-49e7-b3f2-25cdf7ec9f9d None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:33.495384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:33.528563 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4e526-c144-4ffc-81e6-40b9a72827ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:33.529471 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4e526-c144-4ffc-81e6-40b9a72827ee None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:33.529757 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4e526-c144-4ffc-81e6-40b9a72827ee None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:33.530036 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4e526-c144-4ffc-81e6-40b9a72827ee None admin] PATH_INFO: `/v3/users/placement` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:33.538919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4e526-c144-4ffc-81e6-40b9a72827ee None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:33.540365 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4e526-c144-4ffc-81e6-40b9a72827ee None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:33.555358 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-baf4e526-c144-4ffc-81e6-40b9a72827ee None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:30:33.555358 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:33.555358 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:33.555358 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:33.555358 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:33.555358 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:33.555358 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:33.555358 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:33.555358 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:33.555358 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:33.555358 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:33.555358 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:33.555358 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:30:33.555358 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:30:33.555358 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:30:33.555358 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:30:33.562997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:33.562997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:33.562997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:30:33.562997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:30:33.562997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:30:33.562997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:30:33.562997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:33.562997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:33.562997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:33.562997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:33.562997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:33.562997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:33.562997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:33.562997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:33.562997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:33.562997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:33.562997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:33.562997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:33.564338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:33.564338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:33.564338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:30:33.564338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:30:33.564338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:30:33.564338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:30:33.564338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:30:33.564338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:30:33.564338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:30:33.564338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:30:33.564338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 616/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:30:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:33.565142 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb42c10-1490-46ad-9c87-0a3c72db43ad None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:33.667101 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb42c10-1490-46ad-9c87-0a3c72db43ad None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:33.704232 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79a8bc-eea7-4cea-8b3c-7170d61c41f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:33.704881 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79a8bc-eea7-4cea-8b3c-7170d61c41f4 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:33.705165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79a8bc-eea7-4cea-8b3c-7170d61c41f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:33.705439 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79a8bc-eea7-4cea-8b3c-7170d61c41f4 None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:33.706411 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79a8bc-eea7-4cea-8b3c-7170d61c41f4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:33.706700 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79a8bc-eea7-4cea-8b3c-7170d61c41f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:33.708067 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79a8bc-eea7-4cea-8b3c-7170d61c41f4 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:33.726076 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 597/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 23 22:30:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:33.731514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4e526-c144-4ffc-81e6-40b9a72827ee None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:33.839047 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4e526-c144-4ffc-81e6-40b9a72827ee None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:33.874171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf93c26-5581-485e-8d2d-9c7e9eb6e94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:33.874861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf93c26-5581-485e-8d2d-9c7e9eb6e94c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:33.875148 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf93c26-5581-485e-8d2d-9c7e9eb6e94c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:33.876171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf93c26-5581-485e-8d2d-9c7e9eb6e94c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:33.906565 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:33.945384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf93c26-5581-485e-8d2d-9c7e9eb6e94c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:33.946839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf93c26-5581-485e-8d2d-9c7e9eb6e94c None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:34.052552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 617/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:30:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 22:30:34.058464 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79a8bc-eea7-4cea-8b3c-7170d61c41f4 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:34.078791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:34.163134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79a8bc-eea7-4cea-8b3c-7170d61c41f4 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:34.198626 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4ee5c-a853-48e9-a9f0-afe7ecd68a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:34.199429 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4ee5c-a853-48e9-a9f0-afe7ecd68a0f None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:34.199713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4ee5c-a853-48e9-a9f0-afe7ecd68a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:34.199991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4ee5c-a853-48e9-a9f0-afe7ecd68a0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:34.210655 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4ee5c-a853-48e9-a9f0-afe7ecd68a0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:34.212105 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4ee5c-a853-48e9-a9f0-afe7ecd68a0f None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:34.230326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-aed4ee5c-a853-48e9-a9f0-afe7ecd68a0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:34.230326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:34.230326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:34.230326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:34.230326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:34.230326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:34.230326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:34.230326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:34.230326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:34.230326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:34.230326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:34.230326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:34.230326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:30:34.230326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:30:34.230326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:30:34.231347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:30:34.231347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:34.231347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:34.231347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:34.231347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:34.231347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:34.231347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:34.231347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:34.231347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:34.231347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:34.231347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:34.231347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:34.231347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:34.231347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:34.231347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:34.231347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:34.231347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:34.232299 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:30:34.232299 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:30:34.232299 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:34.232299 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:30:34.232299 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 598/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:34.237893 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf93c26-5581-485e-8d2d-9c7e9eb6e94c None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:34.305862 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:34.345186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf93c26-5581-485e-8d2d-9c7e9eb6e94c None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:34.380626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917dbc7-ee0a-49a0-8705-454f65caca21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:34.381525 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917dbc7-ee0a-49a0-8705-454f65caca21 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:34.381812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917dbc7-ee0a-49a0-8705-454f65caca21 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:34.382089 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917dbc7-ee0a-49a0-8705-454f65caca21 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:34.382814 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917dbc7-ee0a-49a0-8705-454f65caca21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:34.383149 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917dbc7-ee0a-49a0-8705-454f65caca21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:34.384511 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917dbc7-ee0a-49a0-8705-454f65caca21 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:34.396932 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 618/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:34.402658 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4ee5c-a853-48e9-a9f0-afe7ecd68a0f None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:34.475535 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:34.509379 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4ee5c-a853-48e9-a9f0-afe7ecd68a0f None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:34.543220 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b65615-b313-45f3-806a-096a4cf04caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:34.544017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b65615-b313-45f3-806a-096a4cf04caa None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:34.544299 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b65615-b313-45f3-806a-096a4cf04caa None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:34.544613 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b65615-b313-45f3-806a-096a4cf04caa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:34.552856 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b65615-b313-45f3-806a-096a4cf04caa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:34.556852 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b65615-b313-45f3-806a-096a4cf04caa None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:34.569863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-b0b65615-b313-45f3-806a-096a4cf04caa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:30:34.569863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:34.569863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:34.569863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:34.569863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:34.569863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:34.569863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:34.569863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:34.569863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:34.569863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:34.569863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:34.569863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:34.569863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:30:34.569863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:30:34.569863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:30:34.570965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:30:34.570965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:34.570965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:34.570965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:34.570965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:34.570965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:34.570965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:34.570965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:34.570965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:34.570965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:34.570965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:34.570965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:34.570965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:34.570965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:34.570965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:34.570965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:34.570965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:34.571880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:30:34.571880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:30:34.571880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:30:34.571880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:30:34.571880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:30:34.571880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:30:34.571880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:30:34.571880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:30:34.571880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 599/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:30:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:34.577055 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917dbc7-ee0a-49a0-8705-454f65caca21 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:34.680451 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917dbc7-ee0a-49a0-8705-454f65caca21 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:34.704334 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:34.716485 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ae079-d13b-4413-a2f1-c6d012993d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:34.717185 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ae079-d13b-4413-a2f1-c6d012993d98 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:34.717470 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ae079-d13b-4413-a2f1-c6d012993d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:34.717744 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ae079-d13b-4413-a2f1-c6d012993d98 None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:34.718577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ae079-d13b-4413-a2f1-c6d012993d98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:34.718920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ae079-d13b-4413-a2f1-c6d012993d98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:34.720500 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ae079-d13b-4413-a2f1-c6d012993d98 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:34.732849 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 619/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:30:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:34.739669 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b65615-b313-45f3-806a-096a4cf04caa None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:34.849081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b65615-b313-45f3-806a-096a4cf04caa None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:34.879728 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:34.885536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8ebe0-8637-4241-b234-bcd5ac1ed99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:34.886192 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8ebe0-8637-4241-b234-bcd5ac1ed99c None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:34.886481 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8ebe0-8637-4241-b234-bcd5ac1ed99c None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:34.886758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8ebe0-8637-4241-b234-bcd5ac1ed99c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:34.887594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8ebe0-8637-4241-b234-bcd5ac1ed99c None admin] RBAC: Adding query filter params (role.id=698ff821321a43d494fe8940cdd08564, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=d3ac44b3f0224e56a15993a2f29c1e29) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:34.887909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8ebe0-8637-4241-b234-bcd5ac1ed99c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:34.889484 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8ebe0-8637-4241-b234-bcd5ac1ed99c None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:34.895474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 600/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:30:34 2022] GET /identity/v3/role_assignments?user.id=d3ac44b3f0224e56a15993a2f29c1e29&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=698ff821321a43d494fe8940cdd08564 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:36.940391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6726e-935e-44fb-bc38-92404f9b9bf6 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:36.942934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6726e-935e-44fb-bc38-92404f9b9bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:36.943246 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6726e-935e-44fb-bc38-92404f9b9bf6 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:36.944681 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 620/1220] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:30:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:30:36.950420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d921de-c180-4ff4-b958-2c383c003aa2 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:36.950928 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d921de-c180-4ff4-b958-2c383c003aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:36.951021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d921de-c180-4ff4-b958-2c383c003aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:36.968340 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-f9d921de-c180-4ff4-b958-2c383c003aa2 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:36.970548 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-f9d921de-c180-4ff4-b958-2c383c003aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:37.043155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 601/1221] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:30:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:30:37.067082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6726e-935e-44fb-bc38-92404f9b9bf6 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:37.173661 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6726e-935e-44fb-bc38-92404f9b9bf6 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:37.208005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43074b4f-feaf-49b2-afe4-3cb9da9c0e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:37.208772 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43074b4f-feaf-49b2-afe4-3cb9da9c0e57 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:37.209063 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43074b4f-feaf-49b2-afe4-3cb9da9c0e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:37.209345 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43074b4f-feaf-49b2-afe4-3cb9da9c0e57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:37.220328 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43074b4f-feaf-49b2-afe4-3cb9da9c0e57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:30:37.228206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-43074b4f-feaf-49b2-afe4-3cb9da9c0e57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:30:37.228206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:37.228206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:37.228206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:37.228206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:37.228206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:37.228206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:37.228206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:37.228206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:37.228206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:37.228206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:37.228206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:37.228206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:30:37.228206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:30:37.228206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:30:37.228206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:30:37.233530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:30:37.233530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:30:37.233530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:30:37.233530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:30:37.233530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:37.233530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:37.233530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:37.233530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:37.233530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:37.233530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:37.233530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:37.233530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:37.233530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:37.233530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:37.233530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:37.233530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:37.234404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:37.234404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:37.234404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:37.234404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:37.234404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:30:37.234404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:30:37.234404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:30:37.234404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:30:37.234404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:30:37.234404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:30:37.234404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:30:37.234404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:30:37.234404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 621/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:30:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:37.236366 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d921de-c180-4ff4-b958-2c383c003aa2 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:37.335777 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d921de-c180-4ff4-b958-2c383c003aa2 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:37.360648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:37.371091 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c978b-edd8-452d-8e4b-400905354b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:37.371746 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c978b-edd8-452d-8e4b-400905354b59 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:37.372035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c978b-edd8-452d-8e4b-400905354b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:37.372308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c978b-edd8-452d-8e4b-400905354b59 None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:37.373100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c978b-edd8-452d-8e4b-400905354b59 None admin] RBAC: Adding query filter params (name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:37.373405 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c978b-edd8-452d-8e4b-400905354b59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:37.374768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c978b-edd8-452d-8e4b-400905354b59 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:37.382531 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 602/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:30:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:37.388330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43074b4f-feaf-49b2-afe4-3cb9da9c0e57 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:37.435276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:37.490903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43074b4f-feaf-49b2-afe4-3cb9da9c0e57 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:37.524984 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a47f46-3e54-436c-9fae-af4c180f459d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:37.525770 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a47f46-3e54-436c-9fae-af4c180f459d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:37.526078 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a47f46-3e54-436c-9fae-af4c180f459d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:37.526359 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a47f46-3e54-436c-9fae-af4c180f459d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:37.537058 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a47f46-3e54-436c-9fae-af4c180f459d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:37.538520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a47f46-3e54-436c-9fae-af4c180f459d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:37.555808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-53a47f46-3e54-436c-9fae-af4c180f459d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:37.555808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:37.555808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:37.555808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:37.555808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:37.555808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:37.555808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:37.555808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:37.555808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:37.555808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:37.555808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:37.555808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:37.555808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:30:37.555808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:30:37.555808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:30:37.556887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:30:37.556887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:37.556887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:37.556887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:37.556887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:37.556887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:37.556887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:37.556887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:37.556887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:37.556887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:37.556887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:37.556887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:37.556887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:37.556887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:37.556887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:37.556887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:37.556887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:37.558016 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:30:37.558016 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:30:37.558016 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:37.558016 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:30:37.558016 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 622/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:30:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:30:37.563335 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c978b-edd8-452d-8e4b-400905354b59 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:37.662795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c978b-edd8-452d-8e4b-400905354b59 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:37.696791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424619f4-938c-4573-9793-c4c926ba8f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:37.697446 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424619f4-938c-4573-9793-c4c926ba8f08 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:37.697733 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424619f4-938c-4573-9793-c4c926ba8f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:37.698010 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424619f4-938c-4573-9793-c4c926ba8f08 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:37.698720 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424619f4-938c-4573-9793-c4c926ba8f08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:37.699028 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424619f4-938c-4573-9793-c4c926ba8f08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:37.700400 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424619f4-938c-4573-9793-c4c926ba8f08 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:37.712410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 603/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:37.718179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a47f46-3e54-436c-9fae-af4c180f459d None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:37.817199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:37.823600 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a47f46-3e54-436c-9fae-af4c180f459d None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:37.856683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880db79d-9caf-4138-903b-a16f42e72383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:37.857856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880db79d-9caf-4138-903b-a16f42e72383 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:37.858145 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880db79d-9caf-4138-903b-a16f42e72383 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:37.858428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880db79d-9caf-4138-903b-a16f42e72383 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:37.866430 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880db79d-9caf-4138-903b-a16f42e72383 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:37.867844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880db79d-9caf-4138-903b-a16f42e72383 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:37.882404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-880db79d-9caf-4138-903b-a16f42e72383 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:30:37.882404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:37.882404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:37.882404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:37.882404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:37.882404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:37.882404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:37.882404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:37.882404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:37.882404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:37.882404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:37.882404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:37.882404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:30:37.882404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:30:37.882404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:30:37.882404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:30:37.883668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:37.883668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:37.883668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:30:37.883668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:30:37.883668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:30:37.883668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:30:37.883668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:37.883668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:37.883668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:37.883668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:37.883668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:37.883668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:37.883668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:37.883668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:37.883668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:37.883668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:37.883668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:37.883668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:37.885048 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:37.885048 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:37.885048 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:30:37.885048 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:30:37.885048 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:30:37.885048 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:30:37.885048 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:30:37.885048 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:30:37.885048 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:30:37.885048 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:30:37.885048 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 623/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:30:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:37.889658 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424619f4-938c-4573-9793-c4c926ba8f08 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:37.936337 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:37.988208 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424619f4-938c-4573-9793-c4c926ba8f08 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:38.020813 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90155b81-d03e-42a1-9b0b-e16b60f3b92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:38.021482 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90155b81-d03e-42a1-9b0b-e16b60f3b92f None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:38.021854 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90155b81-d03e-42a1-9b0b-e16b60f3b92f None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:38.022052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90155b81-d03e-42a1-9b0b-e16b60f3b92f None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:38.022950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90155b81-d03e-42a1-9b0b-e16b60f3b92f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:38.023267 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90155b81-d03e-42a1-9b0b-e16b60f3b92f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:38.024726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90155b81-d03e-42a1-9b0b-e16b60f3b92f None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:38.042141 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 604/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 23 22:30:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:38.048443 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880db79d-9caf-4138-903b-a16f42e72383 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:38.150066 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880db79d-9caf-4138-903b-a16f42e72383 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:38.184043 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c0f48-1235-4013-91e4-8d323fa02633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:38.184867 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c0f48-1235-4013-91e4-8d323fa02633 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:38.185156 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c0f48-1235-4013-91e4-8d323fa02633 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:38.185438 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c0f48-1235-4013-91e4-8d323fa02633 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:38.196131 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c0f48-1235-4013-91e4-8d323fa02633 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:38.197605 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c0f48-1235-4013-91e4-8d323fa02633 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:38.215023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-277c0f48-1235-4013-91e4-8d323fa02633 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:38.215023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:38.215023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:38.215023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:38.215023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:38.215023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:38.215023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:38.215023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:38.215023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:38.215023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:38.215023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:38.215023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:38.215023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:30:38.215023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:30:38.215023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:30:38.215944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:30:38.215944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:38.215944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:38.215944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:38.215944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:38.215944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:38.215944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:38.215944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:38.215944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:38.215944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:38.215944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:38.215944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:38.215944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:38.215944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:38.215944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:38.215944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:38.215944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:38.217110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:30:38.217110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:30:38.217110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:38.217110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:30:38.217110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 624/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:38.222661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90155b81-d03e-42a1-9b0b-e16b60f3b92f None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:38.323242 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90155b81-d03e-42a1-9b0b-e16b60f3b92f None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:38.325678 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:38.357492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fece4e0-d263-4d54-8546-4ab73c67c584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:38.358153 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fece4e0-d263-4d54-8546-4ab73c67c584 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:38.358442 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fece4e0-d263-4d54-8546-4ab73c67c584 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:38.358724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fece4e0-d263-4d54-8546-4ab73c67c584 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:38.359499 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fece4e0-d263-4d54-8546-4ab73c67c584 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:38.359789 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fece4e0-d263-4d54-8546-4ab73c67c584 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:38.361240 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fece4e0-d263-4d54-8546-4ab73c67c584 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:38.372340 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 605/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:38.378409 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c0f48-1235-4013-91e4-8d323fa02633 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:38.387942 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:38.477436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c0f48-1235-4013-91e4-8d323fa02633 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:38.510972 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8348cdd-fbca-4053-9a7a-9847c1f966f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:38.511757 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8348cdd-fbca-4053-9a7a-9847c1f966f5 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:38.512039 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8348cdd-fbca-4053-9a7a-9847c1f966f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:38.512315 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8348cdd-fbca-4053-9a7a-9847c1f966f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:38.519430 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8348cdd-fbca-4053-9a7a-9847c1f966f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:38.520877 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8348cdd-fbca-4053-9a7a-9847c1f966f5 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:38.536369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-a8348cdd-fbca-4053-9a7a-9847c1f966f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:30:38.536369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:38.536369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:38.536369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:38.536369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:38.536369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:38.536369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:38.536369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:38.536369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:38.536369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:38.536369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:38.536369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:38.536369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:30:38.536369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:30:38.536369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:30:38.537459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:30:38.537459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:38.537459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:38.537459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:38.537459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:38.537459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:38.537459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:38.537459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:38.537459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:38.537459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:38.537459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:38.537459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:38.537459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:38.537459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:38.537459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:38.537459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:38.537459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:38.538572 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:30:38.538572 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:30:38.538572 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:30:38.538572 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:30:38.538572 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:30:38.538572 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:30:38.538572 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:30:38.538572 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:30:38.538572 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 625/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:30:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:38.543382 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fece4e0-d263-4d54-8546-4ab73c67c584 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:38.641471 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fece4e0-d263-4d54-8546-4ab73c67c584 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:38.676651 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19e326-32d4-4b77-8940-c8cd4f337c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:38.677351 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19e326-32d4-4b77-8940-c8cd4f337c9c None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:38.677637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19e326-32d4-4b77-8940-c8cd4f337c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:38.677915 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19e326-32d4-4b77-8940-c8cd4f337c9c None admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:38.678735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19e326-32d4-4b77-8940-c8cd4f337c9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:38.679046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19e326-32d4-4b77-8940-c8cd4f337c9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:38.680463 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19e326-32d4-4b77-8940-c8cd4f337c9c None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:38.692524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 606/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:30:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:38.698765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8348cdd-fbca-4053-9a7a-9847c1f966f5 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:38.765341 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:38.799833 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8348cdd-fbca-4053-9a7a-9847c1f966f5 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:38.832279 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0d9a0-946a-4fe1-b46a-849647d45f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:38.833187 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0d9a0-946a-4fe1-b46a-849647d45f0e None admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:38.833473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0d9a0-946a-4fe1-b46a-849647d45f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:38.833752 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0d9a0-946a-4fe1-b46a-849647d45f0e None admin] PATH_INFO: `/v3/projects/14265887fea245a5a37abbb3a116e352/users/d3ac44b3f0224e56a15993a2f29c1e29/roles/698ff821321a43d494fe8940cdd08564` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:38.847949 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0d9a0-946a-4fe1-b46a-849647d45f0e None admin] RBAC: Authorizing `identity:create_grant(project_id=14265887fea245a5a37abbb3a116e352, user_id=d3ac44b3f0224e56a15993a2f29c1e29, role_id=698ff821321a43d494fe8940cdd08564)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:38.849420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0d9a0-946a-4fe1-b46a-849647d45f0e None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:38.914422 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 626/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:30:38 2022] PUT /identity/v3/projects/14265887fea245a5a37abbb3a116e352/users/d3ac44b3f0224e56a15993a2f29c1e29/roles/698ff821321a43d494fe8940cdd08564 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:30:40.945487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3edaf-1a8a-4864-9c4a-9149c7841e4c None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:40.946084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3edaf-1a8a-4864-9c4a-9149c7841e4c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:40.946214 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3edaf-1a8a-4864-9c4a-9149c7841e4c None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:40.947516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 607/1233] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:30:40.952605 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc456d3b-3f66-4e6a-8a10-82fdcfc3bae7 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:40.953045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc456d3b-3f66-4e6a-8a10-82fdcfc3bae7 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:40.953154 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc456d3b-3f66-4e6a-8a10-82fdcfc3bae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:40.969074 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-bc456d3b-3f66-4e6a-8a10-82fdcfc3bae7 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:40.970848 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-bc456d3b-3f66-4e6a-8a10-82fdcfc3bae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:41.039048 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 627/1234] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:30:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:30:41.063659 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3edaf-1a8a-4864-9c4a-9149c7841e4c None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:41.090060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:41.166387 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3edaf-1a8a-4864-9c4a-9149c7841e4c None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:41.199804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417406a7-418b-4fab-a316-20722fd4406f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:41.200532 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417406a7-418b-4fab-a316-20722fd4406f None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:41.200847 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417406a7-418b-4fab-a316-20722fd4406f None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:41.201124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417406a7-418b-4fab-a316-20722fd4406f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:41.211906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417406a7-418b-4fab-a316-20722fd4406f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:30:41.219625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-417406a7-418b-4fab-a316-20722fd4406f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:30:41.219625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:41.219625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:41.219625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:41.219625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:41.219625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:41.219625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:41.219625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:41.219625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:41.219625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:41.219625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:41.219625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:41.219625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:30:41.219625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:30:41.219625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:30:41.219625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:30:41.221501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:30:41.221501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:30:41.221501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:30:41.221501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:30:41.221501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:41.221501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:41.221501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:41.221501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:41.221501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:41.221501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:41.221501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:41.221501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:41.221501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:41.221501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:41.221501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:41.221501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:41.222544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:41.222544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:41.222544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:41.222544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:41.222544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:30:41.222544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:30:41.222544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:30:41.222544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:30:41.222544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:30:41.222544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:30:41.222544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:30:41.222544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:30:41.222544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 608/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 23 22:30:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:41.229565 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc456d3b-3f66-4e6a-8a10-82fdcfc3bae7 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:41.275811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:41.328341 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc456d3b-3f66-4e6a-8a10-82fdcfc3bae7 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:41.361521 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d96df-7168-4c19-a56f-576862ea8b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:41.362182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d96df-7168-4c19-a56f-576862ea8b08 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:41.362472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d96df-7168-4c19-a56f-576862ea8b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:41.362751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d96df-7168-4c19-a56f-576862ea8b08 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:41.363537 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d96df-7168-4c19-a56f-576862ea8b08 None admin] RBAC: Adding query filter params (name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:41.363847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d96df-7168-4c19-a56f-576862ea8b08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:41.365240 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d96df-7168-4c19-a56f-576862ea8b08 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:41.373424 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 628/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 23 22:30:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:41.379309 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417406a7-418b-4fab-a316-20722fd4406f None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:41.464031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:41.479985 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417406a7-418b-4fab-a316-20722fd4406f None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:41.512778 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aad5ec-e236-4631-845f-51ef71c634c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:41.513568 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aad5ec-e236-4631-845f-51ef71c634c2 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:41.513856 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aad5ec-e236-4631-845f-51ef71c634c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:41.514136 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aad5ec-e236-4631-845f-51ef71c634c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:41.524369 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aad5ec-e236-4631-845f-51ef71c634c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:41.525856 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aad5ec-e236-4631-845f-51ef71c634c2 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:41.543780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-25aad5ec-e236-4631-845f-51ef71c634c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:41.543780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:41.543780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:41.543780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:41.543780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:41.543780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:41.543780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:41.543780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:41.543780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:41.543780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:41.543780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:41.543780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:41.543780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:30:41.543780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:30:41.543780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:30:41.544864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:30:41.544864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:41.544864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:41.544864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:41.544864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:41.544864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:41.544864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:41.544864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:41.544864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:41.544864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:41.544864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:41.544864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:41.544864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:41.544864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:41.544864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:41.544864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:41.544864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:41.545990 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:30:41.545990 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:30:41.545990 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:41.545990 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:30:41.545990 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 609/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:41.550884 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d96df-7168-4c19-a56f-576862ea8b08 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:41.650047 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d96df-7168-4c19-a56f-576862ea8b08 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:41.660133 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:41.684617 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ee0ec-1565-42d2-8a0c-380fa1a44a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:41.685277 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ee0ec-1565-42d2-8a0c-380fa1a44a29 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:41.685563 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ee0ec-1565-42d2-8a0c-380fa1a44a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:41.685843 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ee0ec-1565-42d2-8a0c-380fa1a44a29 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:41.686567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ee0ec-1565-42d2-8a0c-380fa1a44a29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:41.686876 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ee0ec-1565-42d2-8a0c-380fa1a44a29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:41.688285 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ee0ec-1565-42d2-8a0c-380fa1a44a29 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:41.699380 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 629/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:41.705440 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aad5ec-e236-4631-845f-51ef71c634c2 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:41.805384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aad5ec-e236-4631-845f-51ef71c634c2 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:41.841833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046cba05-a36a-4b77-90a9-f8aee5478ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:41.842641 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046cba05-a36a-4b77-90a9-f8aee5478ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:41.842961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046cba05-a36a-4b77-90a9-f8aee5478ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:41.843268 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046cba05-a36a-4b77-90a9-f8aee5478ee8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:41.846620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:41.851110 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046cba05-a36a-4b77-90a9-f8aee5478ee8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:41.852538 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046cba05-a36a-4b77-90a9-f8aee5478ee8 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:41.867844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-046cba05-a36a-4b77-90a9-f8aee5478ee8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:30:41.867844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:41.867844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:41.867844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:41.867844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:41.867844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:41.867844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:41.867844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:41.867844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:41.867844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:41.867844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:41.867844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:41.867844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:30:41.867844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:30:41.867844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:30:41.867844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:30:41.868932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:41.868932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:41.868932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:30:41.868932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:30:41.868932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:30:41.868932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:30:41.868932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:41.868932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:41.868932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:41.868932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:41.868932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:41.868932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:41.868932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:41.868932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:41.868932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:41.868932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:41.868932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:41.868932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:41.869875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:41.869875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:41.869875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:30:41.869875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:30:41.869875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:30:41.869875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:30:41.869875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:30:41.869875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:30:41.869875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:30:41.869875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:30:41.869875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 610/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:30:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:41.874930 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ee0ec-1565-42d2-8a0c-380fa1a44a29 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:41.975242 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ee0ec-1565-42d2-8a0c-380fa1a44a29 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:42.007454 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dea26a-9ae7-46e3-91ea-dc4772e6fe0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:42.008123 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dea26a-9ae7-46e3-91ea-dc4772e6fe0a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:42.008403 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dea26a-9ae7-46e3-91ea-dc4772e6fe0a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:42.008707 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dea26a-9ae7-46e3-91ea-dc4772e6fe0a None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:42.009594 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dea26a-9ae7-46e3-91ea-dc4772e6fe0a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:42.009905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dea26a-9ae7-46e3-91ea-dc4772e6fe0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:42.011322 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dea26a-9ae7-46e3-91ea-dc4772e6fe0a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:42.029278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 630/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 23 22:30:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:42.036255 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046cba05-a36a-4b77-90a9-f8aee5478ee8 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:42.132076 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046cba05-a36a-4b77-90a9-f8aee5478ee8 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:42.166132 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8e707-f140-4409-a98d-a18952215530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:42.166776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8e707-f140-4409-a98d-a18952215530 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:42.167066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8e707-f140-4409-a98d-a18952215530 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:42.167344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8e707-f140-4409-a98d-a18952215530 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:42.229244 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:42.235727 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8e707-f140-4409-a98d-a18952215530 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:42.237587 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8e707-f140-4409-a98d-a18952215530 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:42.338372 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 611/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:30:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:30:42.344160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dea26a-9ae7-46e3-91ea-dc4772e6fe0a None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:42.371343 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:42.443906 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dea26a-9ae7-46e3-91ea-dc4772e6fe0a None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:42.477377 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c3fd5-948d-4c26-8c0e-f9a8d456d96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:42.478158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c3fd5-948d-4c26-8c0e-f9a8d456d96b None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:42.478443 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c3fd5-948d-4c26-8c0e-f9a8d456d96b None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:42.478721 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c3fd5-948d-4c26-8c0e-f9a8d456d96b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:42.488567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c3fd5-948d-4c26-8c0e-f9a8d456d96b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:42.490024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c3fd5-948d-4c26-8c0e-f9a8d456d96b None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:42.507573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-7a4c3fd5-948d-4c26-8c0e-f9a8d456d96b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:42.507573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:42.507573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:42.507573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:42.507573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:42.507573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:42.507573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:42.507573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:42.507573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:42.507573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:42.507573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:42.507573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:42.507573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:30:42.507573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:30:42.507573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:30:42.512989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:30:42.512989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:42.512989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:42.512989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:42.512989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:42.512989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:42.512989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:42.512989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:42.512989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:42.512989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:42.512989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:42.512989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:42.512989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:42.512989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:42.512989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:42.512989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:42.512989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:42.514067 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:30:42.514067 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:30:42.514067 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:42.514067 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:30:42.514067 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 631/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:30:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:42.515481 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8e707-f140-4409-a98d-a18952215530 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:42.608850 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:42.614352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8e707-f140-4409-a98d-a18952215530 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:42.647404 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b55e4-cc34-479b-a29d-fbcf8bc17f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:42.648062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b55e4-cc34-479b-a29d-fbcf8bc17f4e None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:42.648345 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b55e4-cc34-479b-a29d-fbcf8bc17f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:42.648651 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b55e4-cc34-479b-a29d-fbcf8bc17f4e None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:42.649375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b55e4-cc34-479b-a29d-fbcf8bc17f4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:42.649683 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b55e4-cc34-479b-a29d-fbcf8bc17f4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:42.651055 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b55e4-cc34-479b-a29d-fbcf8bc17f4e None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:42.662277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 612/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:42.668042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c3fd5-948d-4c26-8c0e-f9a8d456d96b None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:42.747388 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:42.770195 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c3fd5-948d-4c26-8c0e-f9a8d456d96b None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:42.803338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92faee-937c-4765-b217-1427ae457e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:42.804121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92faee-937c-4765-b217-1427ae457e64 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:42.804407 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92faee-937c-4765-b217-1427ae457e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:42.804712 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92faee-937c-4765-b217-1427ae457e64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:42.811397 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f92faee-937c-4765-b217-1427ae457e64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:42.814935 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f92faee-937c-4765-b217-1427ae457e64 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:42.827434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-1f92faee-937c-4765-b217-1427ae457e64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:30:42.827434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:42.827434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:42.827434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:42.827434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:42.827434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:42.827434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:42.827434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:42.827434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:42.827434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:42.827434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:42.827434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:42.827434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:30:42.827434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:30:42.827434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:30:42.828467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:30:42.828467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:42.828467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:42.828467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:42.828467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:42.828467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:42.828467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:42.828467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:42.828467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:42.828467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:42.828467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:42.828467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:42.828467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:42.828467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:42.828467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:42.828467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:42.828467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:42.829840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:30:42.829840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:30:42.829840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:30:42.829840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:30:42.829840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:30:42.829840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:30:42.829840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:30:42.829840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:30:42.829840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 632/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:30:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:42.834578 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b55e4-cc34-479b-a29d-fbcf8bc17f4e None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:42.931348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b55e4-cc34-479b-a29d-fbcf8bc17f4e None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:42.963571 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41dc20-9732-4cea-823a-30544ff32ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:42.964251 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41dc20-9732-4cea-823a-30544ff32ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:42.964535 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41dc20-9732-4cea-823a-30544ff32ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:42.964844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41dc20-9732-4cea-823a-30544ff32ff7 None admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:42.965614 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41dc20-9732-4cea-823a-30544ff32ff7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:42.965920 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41dc20-9732-4cea-823a-30544ff32ff7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:42.967326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41dc20-9732-4cea-823a-30544ff32ff7 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:42.978964 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 613/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:30:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:42.985289 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92faee-937c-4765-b217-1427ae457e64 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:43.084973 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92faee-937c-4765-b217-1427ae457e64 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:43.118749 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89dcb2-ed38-4f47-b69c-79eb38b1bcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:43.119397 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89dcb2-ed38-4f47-b69c-79eb38b1bcb1 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:43.119687 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89dcb2-ed38-4f47-b69c-79eb38b1bcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:43.119973 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89dcb2-ed38-4f47-b69c-79eb38b1bcb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:43.120830 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89dcb2-ed38-4f47-b69c-79eb38b1bcb1 None admin] RBAC: Adding query filter params (role.id=698ff821321a43d494fe8940cdd08564, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=d3ac44b3f0224e56a15993a2f29c1e29) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:43.121146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89dcb2-ed38-4f47-b69c-79eb38b1bcb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:43.122572 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89dcb2-ed38-4f47-b69c-79eb38b1bcb1 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:43.128385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 633/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:30:42 2022] GET /identity/v3/role_assignments?user.id=d3ac44b3f0224e56a15993a2f29c1e29&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=698ff821321a43d494fe8940cdd08564 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:45.180721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa21750-5283-430c-b1af-9d9a058682eb None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:45.181251 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa21750-5283-430c-b1af-9d9a058682eb None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:45.181344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa21750-5283-430c-b1af-9d9a058682eb None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:45.182718 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 614/1247] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:30:45.187898 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c5b44-f449-4a23-aa5c-2ca777b46c4c None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:45.188329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c5b44-f449-4a23-aa5c-2ca777b46c4c None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:45.188453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c5b44-f449-4a23-aa5c-2ca777b46c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:45.193537 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:45.206011 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-0c4c5b44-f449-4a23-aa5c-2ca777b46c4c None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:45.207180 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-0c4c5b44-f449-4a23-aa5c-2ca777b46c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:45.272262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 634/1248] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:30:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:30:45.297211 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa21750-5283-430c-b1af-9d9a058682eb None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:45.316003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:45.398884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa21750-5283-430c-b1af-9d9a058682eb None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:45.432842 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1b32d-01a7-410d-9379-c450b7af59b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:45.433568 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1b32d-01a7-410d-9379-c450b7af59b4 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:45.433863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1b32d-01a7-410d-9379-c450b7af59b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:45.434142 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1b32d-01a7-410d-9379-c450b7af59b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:45.444700 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be1b32d-01a7-410d-9379-c450b7af59b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:30:45.452427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-0be1b32d-01a7-410d-9379-c450b7af59b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:30:45.452427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:45.452427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:45.452427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:45.452427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:45.452427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:45.452427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:45.452427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:45.452427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:45.452427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:45.452427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:45.452427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:45.452427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:30:45.452427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:30:45.452427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:30:45.452427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:30:45.453523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:30:45.453523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:30:45.453523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:30:45.453523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:30:45.453523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:45.453523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:45.453523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:45.453523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:45.453523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:45.453523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:45.453523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:45.453523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:45.453523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:45.453523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:45.453523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:45.453523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:45.455042 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:45.455042 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:45.455042 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:45.455042 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:45.455042 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:30:45.455042 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:30:45.455042 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:30:45.455042 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:30:45.455042 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:30:45.455042 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:30:45.455042 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:30:45.455042 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:30:45.455042 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 615/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:30:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:45.460243 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c5b44-f449-4a23-aa5c-2ca777b46c4c None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:45.565876 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c5b44-f449-4a23-aa5c-2ca777b46c4c None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:45.594667 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:45.599833 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cbfc7a-3b57-4a97-b5ff-e470794add71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:45.600508 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbfc7a-3b57-4a97-b5ff-e470794add71 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:45.600824 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbfc7a-3b57-4a97-b5ff-e470794add71 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:45.601103 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbfc7a-3b57-4a97-b5ff-e470794add71 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:45.601849 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cbfc7a-3b57-4a97-b5ff-e470794add71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:45.602156 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cbfc7a-3b57-4a97-b5ff-e470794add71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:45.603523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cbfc7a-3b57-4a97-b5ff-e470794add71 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:45.611654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 635/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:30:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:45.617407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1b32d-01a7-410d-9379-c450b7af59b4 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:45.697014 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:45.719320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1b32d-01a7-410d-9379-c450b7af59b4 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:45.753277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3b8d4-6cb6-4c71-b215-f385ce294a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:45.754066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3b8d4-6cb6-4c71-b215-f385ce294a1e None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:45.754350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3b8d4-6cb6-4c71-b215-f385ce294a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:45.754626 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3b8d4-6cb6-4c71-b215-f385ce294a1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:45.764701 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3b8d4-6cb6-4c71-b215-f385ce294a1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:45.766140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3b8d4-6cb6-4c71-b215-f385ce294a1e None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:45.783285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-4da3b8d4-6cb6-4c71-b215-f385ce294a1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:45.783285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:45.783285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:45.783285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:45.783285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:45.783285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:45.783285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:45.783285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:45.783285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:45.783285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:45.783285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:45.783285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:45.783285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:30:45.783285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:30:45.783285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:30:45.784552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:30:45.784552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:45.784552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:45.784552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:45.784552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:45.784552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:45.784552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:45.784552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:45.784552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:45.784552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:45.784552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:45.784552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:45.784552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:45.784552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:45.784552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:45.784552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:45.784552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:45.785887 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:30:45.785887 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:30:45.785887 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:45.785887 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:30:45.785887 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 616/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:45.790519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cbfc7a-3b57-4a97-b5ff-e470794add71 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:45.890076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cbfc7a-3b57-4a97-b5ff-e470794add71 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:45.922402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40594b84-a196-44b0-bd93-50f0e8ca0a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:45.923658 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40594b84-a196-44b0-bd93-50f0e8ca0a8a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:45.923939 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40594b84-a196-44b0-bd93-50f0e8ca0a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:45.924210 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40594b84-a196-44b0-bd93-50f0e8ca0a8a None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:45.924948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40594b84-a196-44b0-bd93-50f0e8ca0a8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:45.925254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40594b84-a196-44b0-bd93-50f0e8ca0a8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:45.926612 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40594b84-a196-44b0-bd93-50f0e8ca0a8a None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:45.937970 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 636/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:45.944234 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3b8d4-6cb6-4c71-b215-f385ce294a1e None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:46.042395 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3b8d4-6cb6-4c71-b215-f385ce294a1e None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:46.067622 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:46.075391 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953eeea-0679-4aa6-ad0b-059bc713b219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:46.076162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953eeea-0679-4aa6-ad0b-059bc713b219 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:46.076443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953eeea-0679-4aa6-ad0b-059bc713b219 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:46.076749 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953eeea-0679-4aa6-ad0b-059bc713b219 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:46.084010 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953eeea-0679-4aa6-ad0b-059bc713b219 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:46.085474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953eeea-0679-4aa6-ad0b-059bc713b219 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:46.099829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-f953eeea-0679-4aa6-ad0b-059bc713b219 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:30:46.099829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:46.099829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:46.099829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:46.099829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:46.099829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:46.099829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:46.099829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:46.099829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:46.099829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:46.099829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:46.099829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:46.099829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:30:46.099829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:30:46.099829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:30:46.099829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:30:46.101078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:46.101078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:46.101078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:30:46.101078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:30:46.101078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:30:46.101078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:30:46.101078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:46.101078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:46.101078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:46.101078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:46.101078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:46.101078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:46.101078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:46.101078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:46.101078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:46.101078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:46.101078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:46.101078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:46.102087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:46.102087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:46.102087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:30:46.102087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:30:46.102087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:30:46.102087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:30:46.102087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:30:46.102087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:30:46.102087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:30:46.102087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:30:46.102087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 617/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:30:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:46.107158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40594b84-a196-44b0-bd93-50f0e8ca0a8a None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:46.162962 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:46.206402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40594b84-a196-44b0-bd93-50f0e8ca0a8a None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:46.241236 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb704e-d1a9-43e5-90c9-82164d15a809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:46.241908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb704e-d1a9-43e5-90c9-82164d15a809 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:46.242197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb704e-d1a9-43e5-90c9-82164d15a809 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:46.242479 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb704e-d1a9-43e5-90c9-82164d15a809 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:46.243320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb704e-d1a9-43e5-90c9-82164d15a809 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:46.243630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb704e-d1a9-43e5-90c9-82164d15a809 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:46.245182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb704e-d1a9-43e5-90c9-82164d15a809 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:46.261843 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 637/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 23 22:30:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:46.269374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953eeea-0679-4aa6-ad0b-059bc713b219 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:46.370624 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953eeea-0679-4aa6-ad0b-059bc713b219 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:46.403891 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329537cc-fb35-4b95-ac8c-5bbd8738ecaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:46.404530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329537cc-fb35-4b95-ac8c-5bbd8738ecaf None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:46.404857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329537cc-fb35-4b95-ac8c-5bbd8738ecaf None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:46.405231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329537cc-fb35-4b95-ac8c-5bbd8738ecaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:46.459850 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:46.472481 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329537cc-fb35-4b95-ac8c-5bbd8738ecaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:46.474017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329537cc-fb35-4b95-ac8c-5bbd8738ecaf None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:46.569598 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 618/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:30:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:30:46.575390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb704e-d1a9-43e5-90c9-82164d15a809 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:46.674440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb704e-d1a9-43e5-90c9-82164d15a809 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:46.687145 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:46.707970 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fbc24d-5e0a-4f69-9ef1-fffae007614f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:46.708784 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbc24d-5e0a-4f69-9ef1-fffae007614f None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:46.709072 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbc24d-5e0a-4f69-9ef1-fffae007614f None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:46.709351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbc24d-5e0a-4f69-9ef1-fffae007614f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:46.719416 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fbc24d-5e0a-4f69-9ef1-fffae007614f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:46.721578 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fbc24d-5e0a-4f69-9ef1-fffae007614f None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:46.739333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-15fbc24d-5e0a-4f69-9ef1-fffae007614f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:46.739333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:46.739333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:46.739333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:46.739333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:46.739333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:46.739333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:46.739333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:46.739333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:46.739333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:46.739333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:46.739333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:46.739333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:30:46.739333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:30:46.739333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:30:46.740387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:30:46.740387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:46.740387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:46.740387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:46.740387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:46.740387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:46.740387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:46.740387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:46.740387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:46.740387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:46.740387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:46.740387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:46.740387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:46.740387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:46.740387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:46.740387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:46.740387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:46.741599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:30:46.741599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:30:46.741599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:46.741599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:30:46.741599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 638/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:30:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:46.746757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329537cc-fb35-4b95-ac8c-5bbd8738ecaf None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:46.832243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:46.843592 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329537cc-fb35-4b95-ac8c-5bbd8738ecaf None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:46.875433 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9afafb-e623-4e7f-b890-f2788073a641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:46.876644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9afafb-e623-4e7f-b890-f2788073a641 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:46.876935 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9afafb-e623-4e7f-b890-f2788073a641 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:46.877215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9afafb-e623-4e7f-b890-f2788073a641 None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:46.877922 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9afafb-e623-4e7f-b890-f2788073a641 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:46.878229 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9afafb-e623-4e7f-b890-f2788073a641 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:46.879587 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9afafb-e623-4e7f-b890-f2788073a641 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:46.890417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 619/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:46.897018 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fbc24d-5e0a-4f69-9ef1-fffae007614f None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:46.994489 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fbc24d-5e0a-4f69-9ef1-fffae007614f None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:47.028427 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4895f55-c18c-4628-aae1-71feb7d41e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:47.029367 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4895f55-c18c-4628-aae1-71feb7d41e9c None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:47.029652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4895f55-c18c-4628-aae1-71feb7d41e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:47.029928 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4895f55-c18c-4628-aae1-71feb7d41e9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:47.036690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4895f55-c18c-4628-aae1-71feb7d41e9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:47.038129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4895f55-c18c-4628-aae1-71feb7d41e9c None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:47.051324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-d4895f55-c18c-4628-aae1-71feb7d41e9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:30:47.051324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:47.051324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:47.051324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:47.051324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:47.051324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:47.051324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:47.051324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:47.051324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:47.051324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:47.051324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:47.051324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:47.051324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:30:47.051324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:30:47.051324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:30:47.052390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:30:47.052390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:47.052390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:47.052390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:47.052390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:47.052390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:47.052390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:47.052390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:47.052390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:47.052390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:47.052390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:47.052390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:47.052390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:47.052390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:47.052390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:47.052390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:47.052390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:47.053693 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:30:47.053693 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:30:47.053693 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:30:47.053693 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:30:47.053693 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:30:47.053693 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:30:47.053693 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:30:47.053693 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:30:47.053693 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 639/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:30:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:47.058410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9afafb-e623-4e7f-b890-f2788073a641 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:47.163100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9afafb-e623-4e7f-b890-f2788073a641 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:47.197257 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ca1ec-ca64-46f0-936d-3b0ece2b9b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:47.197917 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ca1ec-ca64-46f0-936d-3b0ece2b9b03 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:47.198208 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ca1ec-ca64-46f0-936d-3b0ece2b9b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:47.198502 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ca1ec-ca64-46f0-936d-3b0ece2b9b03 None admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:47.199265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ca1ec-ca64-46f0-936d-3b0ece2b9b03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:47.199576 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ca1ec-ca64-46f0-936d-3b0ece2b9b03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:47.201074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ca1ec-ca64-46f0-936d-3b0ece2b9b03 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:47.213182 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 620/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:30:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:47.219323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4895f55-c18c-4628-aae1-71feb7d41e9c None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:47.245192 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:47.321265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4895f55-c18c-4628-aae1-71feb7d41e9c None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:47.355644 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc72665b-1d64-4fe6-8434-f8e8bc984e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:47.356289 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72665b-1d64-4fe6-8434-f8e8bc984e2b None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:47.356614 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72665b-1d64-4fe6-8434-f8e8bc984e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:47.356998 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72665b-1d64-4fe6-8434-f8e8bc984e2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:47.357863 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc72665b-1d64-4fe6-8434-f8e8bc984e2b None admin] RBAC: Adding query filter params (role.id=6ff7708b59984b9287cc81d46a57b4b4, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=d3ac44b3f0224e56a15993a2f29c1e29) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:47.358175 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc72665b-1d64-4fe6-8434-f8e8bc984e2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:47.359604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc72665b-1d64-4fe6-8434-f8e8bc984e2b None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:47.365804 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 640/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:30:47 2022] GET /identity/v3/role_assignments?user.id=d3ac44b3f0224e56a15993a2f29c1e29&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=6ff7708b59984b9287cc81d46a57b4b4 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:49.382189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b726791-2624-4715-b45c-cb0dd62120c9 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:49.382711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b726791-2624-4715-b45c-cb0dd62120c9 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:49.382832 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b726791-2624-4715-b45c-cb0dd62120c9 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:49.384136 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 621/1261] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:30:49.389520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955b51e-d945-4d9c-b4cf-332c77cc4dbe None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:49.393548 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955b51e-d945-4d9c-b4cf-332c77cc4dbe None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:49.393548 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955b51e-d945-4d9c-b4cf-332c77cc4dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:49.406742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-c955b51e-d945-4d9c-b4cf-332c77cc4dbe None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:49.409092 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-c955b51e-d945-4d9c-b4cf-332c77cc4dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:49.473768 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 641/1262] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:30:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:30:49.496181 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b726791-2624-4715-b45c-cb0dd62120c9 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:49.506940 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:49.596205 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b726791-2624-4715-b45c-cb0dd62120c9 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:49.630006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549d15b-81e5-47a3-b0f1-fec6a76b4d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:49.630795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549d15b-81e5-47a3-b0f1-fec6a76b4d83 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:49.631084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549d15b-81e5-47a3-b0f1-fec6a76b4d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:49.631407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549d15b-81e5-47a3-b0f1-fec6a76b4d83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:49.641367 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549d15b-81e5-47a3-b0f1-fec6a76b4d83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:30:49.649797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-7549d15b-81e5-47a3-b0f1-fec6a76b4d83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:30:49.649797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:49.649797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:49.649797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:49.649797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:49.649797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:49.649797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:49.649797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:49.649797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:49.649797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:49.649797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:49.649797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:49.649797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:30:49.649797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:30:49.649797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:30:49.649797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:30:49.650916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:30:49.650916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:30:49.650916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:30:49.650916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:30:49.650916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:49.650916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:49.650916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:49.650916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:49.650916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:49.650916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:49.650916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:49.650916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:49.650916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:49.650916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:49.650916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:49.650916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:49.651973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:49.651973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:49.651973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:49.651973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:49.651973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:30:49.651973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:30:49.651973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:30:49.651973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:30:49.651973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:30:49.651973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:30:49.651973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:30:49.651973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:30:49.651973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 622/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:30:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:49.657911 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955b51e-d945-4d9c-b4cf-332c77cc4dbe None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:49.676945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:49.757663 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955b51e-d945-4d9c-b4cf-332c77cc4dbe None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:49.791029 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad2994-892e-4a7d-b461-5f46e0b1e5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:49.791709 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad2994-892e-4a7d-b461-5f46e0b1e5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:49.791995 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad2994-892e-4a7d-b461-5f46e0b1e5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:49.792274 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad2994-892e-4a7d-b461-5f46e0b1e5e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:49.793050 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad2994-892e-4a7d-b461-5f46e0b1e5e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:49.793359 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad2994-892e-4a7d-b461-5f46e0b1e5e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:49.794735 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad2994-892e-4a7d-b461-5f46e0b1e5e5 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:49.803232 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 642/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:30:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:49.809192 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549d15b-81e5-47a3-b0f1-fec6a76b4d83 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:49.876323 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:49.909023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549d15b-81e5-47a3-b0f1-fec6a76b4d83 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:49.942391 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfd7ac-5d8a-4b16-bad7-551ff38d3261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:49.943184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfd7ac-5d8a-4b16-bad7-551ff38d3261 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:49.943473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfd7ac-5d8a-4b16-bad7-551ff38d3261 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:49.943755 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfd7ac-5d8a-4b16-bad7-551ff38d3261 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:49.955159 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfd7ac-5d8a-4b16-bad7-551ff38d3261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:49.956676 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfd7ac-5d8a-4b16-bad7-551ff38d3261 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:49.974991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-4bcfd7ac-5d8a-4b16-bad7-551ff38d3261 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:49.974991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:49.974991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:49.974991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:49.974991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:49.974991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:49.974991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:49.974991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:49.974991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:49.974991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:49.974991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:49.974991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:49.974991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:30:49.974991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:30:49.974991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:30:49.976072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:30:49.976072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:49.976072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:49.976072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:49.976072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:49.976072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:49.976072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:49.976072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:49.976072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:49.976072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:49.976072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:49.976072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:49.976072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:49.976072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:49.976072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:49.976072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:49.976072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:49.977140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:30:49.977140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:30:49.977140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:49.977140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:30:49.977140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 623/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:30:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:49.982264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad2994-892e-4a7d-b461-5f46e0b1e5e5 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:50.062465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:50.082016 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad2994-892e-4a7d-b461-5f46e0b1e5e5 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:50.115580 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59e383-6071-4cdf-86a6-3a3c30b24cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:50.116244 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59e383-6071-4cdf-86a6-3a3c30b24cdc None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:50.116562 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59e383-6071-4cdf-86a6-3a3c30b24cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:50.116930 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59e383-6071-4cdf-86a6-3a3c30b24cdc None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:50.117717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59e383-6071-4cdf-86a6-3a3c30b24cdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:50.118235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59e383-6071-4cdf-86a6-3a3c30b24cdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:50.119914 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59e383-6071-4cdf-86a6-3a3c30b24cdc None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:50.132073 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 643/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:30:50.137661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfd7ac-5d8a-4b16-bad7-551ff38d3261 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:50.237714 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfd7ac-5d8a-4b16-bad7-551ff38d3261 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:50.259746 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:50.274219 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b10d0-bd49-45b6-adb9-dc202013073f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:50.275024 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b10d0-bd49-45b6-adb9-dc202013073f None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:50.275634 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b10d0-bd49-45b6-adb9-dc202013073f None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:50.275634 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b10d0-bd49-45b6-adb9-dc202013073f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:50.283438 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b10d0-bd49-45b6-adb9-dc202013073f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:50.286466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b10d0-bd49-45b6-adb9-dc202013073f None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:50.300530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-b24b10d0-bd49-45b6-adb9-dc202013073f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:30:50.300530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:50.300530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:50.300530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:50.300530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:50.300530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:50.300530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:50.300530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:50.300530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:50.300530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:50.300530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:50.300530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:50.300530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:30:50.300530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:30:50.300530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:30:50.300530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:30:50.301775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:50.301775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:50.301775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:30:50.301775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:30:50.301775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:30:50.301775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:30:50.301775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:50.301775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:50.301775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:50.301775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:50.301775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:50.301775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:50.301775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:50.301775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:50.301775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:50.301775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:50.301775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:50.301775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:50.302941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:50.302941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:50.302941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:30:50.302941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:30:50.302941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:30:50.302941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:30:50.302941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:30:50.302941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:30:50.302941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:30:50.302941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:30:50.302941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 624/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:30:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:50.307678 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59e383-6071-4cdf-86a6-3a3c30b24cdc None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:50.407724 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59e383-6071-4cdf-86a6-3a3c30b24cdc None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:50.441463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ccf9e-87b3-4360-a0a0-71e154c59d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:50.442125 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ccf9e-87b3-4360-a0a0-71e154c59d89 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:50.442412 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ccf9e-87b3-4360-a0a0-71e154c59d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:50.442693 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ccf9e-87b3-4360-a0a0-71e154c59d89 None admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:50.443572 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ccf9e-87b3-4360-a0a0-71e154c59d89 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:50.443886 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ccf9e-87b3-4360-a0a0-71e154c59d89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:50.445338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ccf9e-87b3-4360-a0a0-71e154c59d89 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:50.454877 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:50.462889 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 644/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 23 22:30:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:50.468594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b10d0-bd49-45b6-adb9-dc202013073f None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:50.568036 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b10d0-bd49-45b6-adb9-dc202013073f None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:50.602292 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd7c32-18e8-4aaa-9b3e-a8e94f4f19f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:50.603128 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd7c32-18e8-4aaa-9b3e-a8e94f4f19f3 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:50.603414 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd7c32-18e8-4aaa-9b3e-a8e94f4f19f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:50.603695 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd7c32-18e8-4aaa-9b3e-a8e94f4f19f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:50.614720 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd7c32-18e8-4aaa-9b3e-a8e94f4f19f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:50.616166 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd7c32-18e8-4aaa-9b3e-a8e94f4f19f3 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:50.633041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-c1cd7c32-18e8-4aaa-9b3e-a8e94f4f19f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:50.633041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:50.633041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:50.633041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:50.633041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:50.633041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:50.633041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:50.633041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:50.633041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:50.633041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:50.633041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:50.633041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:50.633041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:30:50.633041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:30:50.633041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:30:50.634095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:30:50.634095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:50.634095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:50.634095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:50.634095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:50.634095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:50.634095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:50.634095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:50.634095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:50.634095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:50.634095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:50.634095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:50.634095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:50.634095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:50.634095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:50.634095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:50.634095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:50.635183 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:30:50.635183 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:30:50.635183 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:50.635183 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:30:50.635183 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 625/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:30:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:50.640107 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ccf9e-87b3-4360-a0a0-71e154c59d89 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:50.745753 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ccf9e-87b3-4360-a0a0-71e154c59d89 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:50.778839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1ca01-0007-4cc3-84d3-263a13929f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:50.779498 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1ca01-0007-4cc3-84d3-263a13929f69 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:50.779884 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1ca01-0007-4cc3-84d3-263a13929f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:50.780110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1ca01-0007-4cc3-84d3-263a13929f69 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:50.780808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1ca01-0007-4cc3-84d3-263a13929f69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:50.781113 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1ca01-0007-4cc3-84d3-263a13929f69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:50.783084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1ca01-0007-4cc3-84d3-263a13929f69 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:50.794230 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 645/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:50.800502 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd7c32-18e8-4aaa-9b3e-a8e94f4f19f3 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:50.833255 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:50.901488 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd7c32-18e8-4aaa-9b3e-a8e94f4f19f3 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:50.936702 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5742eb-6a5e-40fb-8066-d74922c86935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:50.937547 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5742eb-6a5e-40fb-8066-d74922c86935 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:50.937856 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5742eb-6a5e-40fb-8066-d74922c86935 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:50.938136 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5742eb-6a5e-40fb-8066-d74922c86935 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:50.944793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5742eb-6a5e-40fb-8066-d74922c86935 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:50.946231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5742eb-6a5e-40fb-8066-d74922c86935 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:50.959966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-cb5742eb-6a5e-40fb-8066-d74922c86935 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:30:50.959966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:50.959966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:50.959966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:50.959966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:50.959966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:50.959966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:50.959966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:50.959966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:50.959966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:50.959966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:50.959966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:50.959966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:30:50.959966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:30:50.959966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:30:50.962943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:30:50.962943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:50.962943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:50.962943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:50.962943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:50.962943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:50.962943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:50.962943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:50.962943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:50.962943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:50.962943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:50.962943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:50.962943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:50.962943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:50.962943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:50.962943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:50.962943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:50.964057 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:30:50.964057 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:30:50.964057 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:30:50.964057 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:30:50.964057 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:30:50.964057 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:30:50.964057 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:30:50.964057 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:30:50.964057 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 626/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:30:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:50.967016 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1ca01-0007-4cc3-84d3-263a13929f69 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:51.025054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:51.065727 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1ca01-0007-4cc3-84d3-263a13929f69 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:51.098251 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9415a-ad2f-4775-8479-bee4da8bf7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:51.098904 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9415a-ad2f-4775-8479-bee4da8bf7cd None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:51.099185 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9415a-ad2f-4775-8479-bee4da8bf7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:51.099459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9415a-ad2f-4775-8479-bee4da8bf7cd None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:51.100346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9415a-ad2f-4775-8479-bee4da8bf7cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:51.100685 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9415a-ad2f-4775-8479-bee4da8bf7cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:51.102110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9415a-ad2f-4775-8479-bee4da8bf7cd None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:51.114264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 646/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:30:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:51.120263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5742eb-6a5e-40fb-8066-d74922c86935 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:51.210927 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:51.219528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5742eb-6a5e-40fb-8066-d74922c86935 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:51.252601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8999152-246a-45a8-ab8e-04a75ca07948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:51.253469 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8999152-246a-45a8-ab8e-04a75ca07948 None admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:51.253750 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8999152-246a-45a8-ab8e-04a75ca07948 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:51.254023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8999152-246a-45a8-ab8e-04a75ca07948 None admin] PATH_INFO: `/v3/projects/14265887fea245a5a37abbb3a116e352/users/d3ac44b3f0224e56a15993a2f29c1e29/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:51.255936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8999152-246a-45a8-ab8e-04a75ca07948 None admin] RBAC: Authorizing `identity:create_grant(project_id=14265887fea245a5a37abbb3a116e352, user_id=d3ac44b3f0224e56a15993a2f29c1e29, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:51.257394 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8999152-246a-45a8-ab8e-04a75ca07948 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:51.353278 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 627/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 23 22:30:51 2022] PUT /identity/v3/projects/14265887fea245a5a37abbb3a116e352/users/d3ac44b3f0224e56a15993a2f29c1e29/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:30:53.399732 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd7759-0b43-4134-b6f9-aa16252783b3 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:53.400328 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd7759-0b43-4134-b6f9-aa16252783b3 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:53.400328 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd7759-0b43-4134-b6f9-aa16252783b3 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:53.401942 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 647/1274] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:30:53.408340 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf094c62-227a-4d88-9993-59df465008d5 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:53.409195 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf094c62-227a-4d88-9993-59df465008d5 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:53.409295 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf094c62-227a-4d88-9993-59df465008d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:53.427022 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-bf094c62-227a-4d88-9993-59df465008d5 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:53.428253 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-bf094c62-227a-4d88-9993-59df465008d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:53.509309 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 628/1275] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:30:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:30:53.535079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd7759-0b43-4134-b6f9-aa16252783b3 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:53.647930 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd7759-0b43-4134-b6f9-aa16252783b3 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:53.669246 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:53.686206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b3753-46e0-4a64-b993-2d4a25e3f94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:53.687003 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b3753-46e0-4a64-b993-2d4a25e3f94a None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:53.687289 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b3753-46e0-4a64-b993-2d4a25e3f94a None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:53.687570 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b3753-46e0-4a64-b993-2d4a25e3f94a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:53.698192 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b3753-46e0-4a64-b993-2d4a25e3f94a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:30:53.705946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-1e1b3753-46e0-4a64-b993-2d4a25e3f94a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:30:53.705946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:53.705946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:53.705946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:53.705946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:53.705946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:53.705946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:53.705946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:53.705946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:53.705946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:53.705946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:53.705946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:53.705946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:30:53.705946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:30:53.705946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:30:53.705946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:30:53.706961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:30:53.706961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:30:53.706961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:30:53.706961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:30:53.706961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:53.706961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:53.706961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:53.706961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:53.706961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:53.706961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:53.706961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:53.706961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:53.706961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:53.706961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:53.706961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:53.706961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:53.707837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:53.707837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:53.707837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:53.707837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:53.707837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:30:53.707837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:30:53.707837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:30:53.707837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:30:53.707837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:30:53.707837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:30:53.707837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:30:53.707837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:30:53.707837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 648/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 23 22:30:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:53.713309 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf094c62-227a-4d88-9993-59df465008d5 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:53.799106 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:53.816810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf094c62-227a-4d88-9993-59df465008d5 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:53.850444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a31f1-9837-4e87-82b6-07111559bcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:53.851106 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a31f1-9837-4e87-82b6-07111559bcbf None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:53.851393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a31f1-9837-4e87-82b6-07111559bcbf None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:53.851671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a31f1-9837-4e87-82b6-07111559bcbf None admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:53.852411 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a31f1-9837-4e87-82b6-07111559bcbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:53.852782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a31f1-9837-4e87-82b6-07111559bcbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:53.854155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a31f1-9837-4e87-82b6-07111559bcbf None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:53.862675 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 629/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 23 22:30:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:53.868528 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b3753-46e0-4a64-b993-2d4a25e3f94a None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:53.967377 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b3753-46e0-4a64-b993-2d4a25e3f94a None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:54.000958 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87e0f0-d3c4-423f-84e0-461338a79364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:54.001746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87e0f0-d3c4-423f-84e0-461338a79364 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:54.002041 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87e0f0-d3c4-423f-84e0-461338a79364 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:54.002324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87e0f0-d3c4-423f-84e0-461338a79364 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:54.012841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87e0f0-d3c4-423f-84e0-461338a79364 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:54.014327 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87e0f0-d3c4-423f-84e0-461338a79364 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:54.031549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-0a87e0f0-d3c4-423f-84e0-461338a79364 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:54.031549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:54.031549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:54.031549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:54.031549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:54.031549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:54.031549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:54.031549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:54.031549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:54.031549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:54.031549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:54.031549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:54.031549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:30:54.031549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:30:54.031549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:30:54.035110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:30:54.035110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:54.035110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:54.035110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:54.035110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:54.035110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:54.035110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:54.035110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:54.035110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:54.035110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:54.035110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:54.035110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:54.035110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:54.035110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:54.035110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:54.035110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:54.035110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:54.036214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:30:54.036214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:30:54.036214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:54.036214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:30:54.036214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 649/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:30:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:54.039609 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a31f1-9837-4e87-82b6-07111559bcbf None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:54.136654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a31f1-9837-4e87-82b6-07111559bcbf None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:54.169767 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d1c16-1976-449c-a63b-2024084023bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:54.170417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d1c16-1976-449c-a63b-2024084023bd None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:54.170699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d1c16-1976-449c-a63b-2024084023bd None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:54.170974 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d1c16-1976-449c-a63b-2024084023bd None admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:54.171748 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d1c16-1976-449c-a63b-2024084023bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:54.172033 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d1c16-1976-449c-a63b-2024084023bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:54.173444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d1c16-1976-449c-a63b-2024084023bd None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:54.184729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 630/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:54.190506 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87e0f0-d3c4-423f-84e0-461338a79364 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:54.221249 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:54.287530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87e0f0-d3c4-423f-84e0-461338a79364 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:54.320446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023df5fb-401b-4a19-88f9-5eb124e4deaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:54.321269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023df5fb-401b-4a19-88f9-5eb124e4deaf None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:54.321552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023df5fb-401b-4a19-88f9-5eb124e4deaf None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:54.321830 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023df5fb-401b-4a19-88f9-5eb124e4deaf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:54.329782 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023df5fb-401b-4a19-88f9-5eb124e4deaf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:54.335268 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023df5fb-401b-4a19-88f9-5eb124e4deaf None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:54.346402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-023df5fb-401b-4a19-88f9-5eb124e4deaf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:30:54.346402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:54.346402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:54.346402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:54.346402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:54.346402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:54.346402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:54.346402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:54.346402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:54.346402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:54.346402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:54.346402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:54.346402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:30:54.346402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:30:54.346402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:30:54.346402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:30:54.347514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:54.347514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:54.347514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:30:54.347514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:30:54.347514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:30:54.347514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:30:54.347514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:54.347514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:54.347514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:54.347514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:54.347514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:54.347514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:54.347514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:54.347514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:54.347514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:54.347514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:54.347514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:54.347514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:54.348766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:54.348766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:54.348766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:30:54.348766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:30:54.348766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:30:54.348766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:30:54.348766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:30:54.348766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:30:54.348766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:30:54.348766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:30:54.348766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 650/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:30:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:54.353434 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d1c16-1976-449c-a63b-2024084023bd None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:54.357405 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:54.452226 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d1c16-1976-449c-a63b-2024084023bd None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:54.484807 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e39f7-68ab-41d8-aa51-cbcd3d68dd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:54.485493 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e39f7-68ab-41d8-aa51-cbcd3d68dd48 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:54.485770 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e39f7-68ab-41d8-aa51-cbcd3d68dd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:54.486046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e39f7-68ab-41d8-aa51-cbcd3d68dd48 None admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:54.486875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e39f7-68ab-41d8-aa51-cbcd3d68dd48 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:54.487182 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e39f7-68ab-41d8-aa51-cbcd3d68dd48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:54.488659 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e39f7-68ab-41d8-aa51-cbcd3d68dd48 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:54.505920 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 631/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 23 22:30:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:54.511707 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023df5fb-401b-4a19-88f9-5eb124e4deaf None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:54.603197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:54.613100 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023df5fb-401b-4a19-88f9-5eb124e4deaf None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:54.646188 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d361ba-fbaf-44af-889a-cce12ced3964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:54.646835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d361ba-fbaf-44af-889a-cce12ced3964 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:54.647122 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d361ba-fbaf-44af-889a-cce12ced3964 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:54.647402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d361ba-fbaf-44af-889a-cce12ced3964 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:54.713595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d361ba-fbaf-44af-889a-cce12ced3964 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:54.715058 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d361ba-fbaf-44af-889a-cce12ced3964 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:54.802984 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:54.812939 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 651/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 23 22:30:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:30:54.818831 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e39f7-68ab-41d8-aa51-cbcd3d68dd48 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:54.879837 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:54.918687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e39f7-68ab-41d8-aa51-cbcd3d68dd48 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:54.953149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b5dcf-dbff-4d52-8b12-ccfbd790def7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:54.954069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b5dcf-dbff-4d52-8b12-ccfbd790def7 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:54.954506 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b5dcf-dbff-4d52-8b12-ccfbd790def7 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:54.954791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b5dcf-dbff-4d52-8b12-ccfbd790def7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:54.965345 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b5dcf-dbff-4d52-8b12-ccfbd790def7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:54.966791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b5dcf-dbff-4d52-8b12-ccfbd790def7 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:54.983998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-927b5dcf-dbff-4d52-8b12-ccfbd790def7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:54.983998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:54.983998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:54.983998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:54.983998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:54.983998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:54.983998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:54.983998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:54.983998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:54.983998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:54.983998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:54.983998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:54.983998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:30:54.983998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:30:54.983998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:30:54.985591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:30:54.985591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:54.985591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:54.985591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:54.985591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:54.985591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:54.985591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:54.985591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:54.985591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:54.985591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:54.985591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:54.985591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:54.985591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:54.985591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:54.985591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:54.985591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:54.985591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:54.986700 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:30:54.986700 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:30:54.986700 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:30:54.986700 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:30:54.986700 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 632/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:30:54.991665 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d361ba-fbaf-44af-889a-cce12ced3964 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:55.088979 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d361ba-fbaf-44af-889a-cce12ced3964 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:55.122245 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abca262-dbe0-4060-83f4-bf2a2da52822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:55.122894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abca262-dbe0-4060-83f4-bf2a2da52822 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:55.123181 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abca262-dbe0-4060-83f4-bf2a2da52822 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:55.123450 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abca262-dbe0-4060-83f4-bf2a2da52822 None admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:55.124170 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abca262-dbe0-4060-83f4-bf2a2da52822 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:55.124471 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abca262-dbe0-4060-83f4-bf2a2da52822 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:55.125862 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abca262-dbe0-4060-83f4-bf2a2da52822 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:55.136663 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 652/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 23 22:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:55.142465 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b5dcf-dbff-4d52-8b12-ccfbd790def7 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:55.242417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b5dcf-dbff-4d52-8b12-ccfbd790def7 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:55.255762 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:55.275299 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49fd0c-345a-454f-804a-fecc1fadc7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:55.276080 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49fd0c-345a-454f-804a-fecc1fadc7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:55.276357 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49fd0c-345a-454f-804a-fecc1fadc7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:55.276653 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49fd0c-345a-454f-804a-fecc1fadc7a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:55.284164 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49fd0c-345a-454f-804a-fecc1fadc7a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:55.285633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49fd0c-345a-454f-804a-fecc1fadc7a6 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:55.301677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application [None req-8d49fd0c-345a-454f-804a-fecc1fadc7a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:30:55.301677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:55.301677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:55.301677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:55.301677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:55.301677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:55.301677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:55.301677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:55.301677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:55.301677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:55.301677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:55.301677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:55.301677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:30:55.301677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:30:55.301677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:30:55.302726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:30:55.302726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:55.302726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:55.302726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:55.302726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:55.302726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:55.302726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:55.302726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:55.302726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:55.302726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:55.302726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:55.302726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:55.302726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:55.302726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:55.302726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:55.302726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:55.302726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:55.303640 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:30:55.303640 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:30:55.303640 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:30:55.303640 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:30:55.303640 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:30:55.303640 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:30:55.303640 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:30:55.303640 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: ERROR keystone.server.flask.application Jun 23 22:30:55.304375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 633/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 23 22:30:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:55.310001 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abca262-dbe0-4060-83f4-bf2a2da52822 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:55.359444 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:55.409331 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abca262-dbe0-4060-83f4-bf2a2da52822 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:55.442159 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f61ad-8d28-485b-8948-61e0401ce179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:55.442853 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f61ad-8d28-485b-8948-61e0401ce179 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:55.443161 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f61ad-8d28-485b-8948-61e0401ce179 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:55.443429 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f61ad-8d28-485b-8948-61e0401ce179 None admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:55.444216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f61ad-8d28-485b-8948-61e0401ce179 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:55.444527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f61ad-8d28-485b-8948-61e0401ce179 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:55.445971 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f61ad-8d28-485b-8948-61e0401ce179 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:55.458423 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 653/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 23 22:30:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:55.464519 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49fd0c-345a-454f-804a-fecc1fadc7a6 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:55.562862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49fd0c-345a-454f-804a-fecc1fadc7a6 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:55.596421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee62d6-ea33-4cc1-a21b-436256187ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:55.597124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee62d6-ea33-4cc1-a21b-436256187ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:55.597403 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee62d6-ea33-4cc1-a21b-436256187ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:55.597682 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee62d6-ea33-4cc1-a21b-436256187ac3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:55.598501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ee62d6-ea33-4cc1-a21b-436256187ac3 None admin] RBAC: Adding query filter params (role.id=6ff7708b59984b9287cc81d46a57b4b4, scope.project.id=14265887fea245a5a37abbb3a116e352, user.id=d3ac44b3f0224e56a15993a2f29c1e29) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:55.598813 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ee62d6-ea33-4cc1-a21b-436256187ac3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:55.600242 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ee62d6-ea33-4cc1-a21b-436256187ac3 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:55.606134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 634/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 23 22:30:55 2022] GET /identity/v3/role_assignments?user.id=d3ac44b3f0224e56a15993a2f29c1e29&scope.project.id=14265887fea245a5a37abbb3a116e352&role.id=6ff7708b59984b9287cc81d46a57b4b4 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:57.664715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691bfcb-d836-49ae-9224-1028e5ab54e4 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:57.665219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691bfcb-d836-49ae-9224-1028e5ab54e4 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:57.665219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691bfcb-d836-49ae-9224-1028e5ab54e4 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:57.666624 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 654/1288] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:30:57.671910 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24b1c1-b3f8-446a-8ab8-88b7e9092ba5 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:57.672866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24b1c1-b3f8-446a-8ab8-88b7e9092ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:57.673001 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24b1c1-b3f8-446a-8ab8-88b7e9092ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:57.690052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-2f24b1c1-b3f8-446a-8ab8-88b7e9092ba5 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:57.693934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-2f24b1c1-b3f8-446a-8ab8-88b7e9092ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:57.720095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:57.756689 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 635/1289] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:30:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:30:57.779423 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691bfcb-d836-49ae-9224-1028e5ab54e4 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:57.880403 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691bfcb-d836-49ae-9224-1028e5ab54e4 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:57.890760 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:57.913664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b45a6-2989-49eb-8d27-39a4ff60ae0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:57.914456 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b45a6-2989-49eb-8d27-39a4ff60ae0b None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:57.914786 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b45a6-2989-49eb-8d27-39a4ff60ae0b None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:57.915013 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b45a6-2989-49eb-8d27-39a4ff60ae0b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:57.915662 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b45a6-2989-49eb-8d27-39a4ff60ae0b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:57.917139 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b45a6-2989-49eb-8d27-39a4ff60ae0b None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:57.930264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-cc8b45a6-2989-49eb-8d27-39a4ff60ae0b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 22:30:57.930264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:57.930264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:30:57.930264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:30:57.930264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:30:57.930264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:30:57.930264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:30:57.930264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:30:57.930264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 22:30:57.930264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:30:57.930264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:30:57.930264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:30:57.930264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:30:57.930264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:30:57.930264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:30:57.930264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:30:57.930264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:30:57.937391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:30:57.937391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:30:57.937391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:30:57.937391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:30:57.937391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:30:57.937391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:30:57.937391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:30:57.937391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:30:57.937391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:30:57.937391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:30:57.937391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:30:57.937391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:30:57.937391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:30:57.937391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:30:57.937391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:30:57.937391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:30:57.939524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:30:57.939524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:30:57.939524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:30:57.939524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:30:57.939524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:30:57.939524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:30:57.939524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:30:57.939524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:30:57.939524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:30:57.939524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:30:57.939524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:30:57.939524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:30:57.939524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 22:30:57.939524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:30:57.939524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 655/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 23 22:30:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:30:57.940541 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24b1c1-b3f8-446a-8ab8-88b7e9092ba5 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:58.035890 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24b1c1-b3f8-446a-8ab8-88b7e9092ba5 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:58.069210 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc1751-6bf7-4ffd-a0b8-11e2512fdf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:58.069863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc1751-6bf7-4ffd-a0b8-11e2512fdf30 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:58.070148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc1751-6bf7-4ffd-a0b8-11e2512fdf30 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:58.070427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc1751-6bf7-4ffd-a0b8-11e2512fdf30 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:58.071124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc1751-6bf7-4ffd-a0b8-11e2512fdf30 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:58.071429 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc1751-6bf7-4ffd-a0b8-11e2512fdf30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:58.072952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc1751-6bf7-4ffd-a0b8-11e2512fdf30 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:58.077752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 636/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Jun 23 22:30:57 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:30:58.083947 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b45a6-2989-49eb-8d27-39a4ff60ae0b None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:58.184534 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b45a6-2989-49eb-8d27-39a4ff60ae0b None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:58.216365 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d2d3c-8f5c-4faa-a07c-0ca79b629125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:58.217046 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d2d3c-8f5c-4faa-a07c-0ca79b629125 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:58.217333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d2d3c-8f5c-4faa-a07c-0ca79b629125 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:58.217614 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d2d3c-8f5c-4faa-a07c-0ca79b629125 None admin] PATH_INFO: `/v3/services` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:58.218317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d2d3c-8f5c-4faa-a07c-0ca79b629125 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:30:58.218622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d2d3c-8f5c-4faa-a07c-0ca79b629125 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:58.220806 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d2d3c-8f5c-4faa-a07c-0ca79b629125 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:58.225192 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 656/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Jun 23 22:30:58 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:31:00.311795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0767e0-7477-4802-89ba-7b29975dc1e3 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:31:00.312319 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0767e0-7477-4802-89ba-7b29975dc1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:31:00.312422 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0767e0-7477-4802-89ba-7b29975dc1e3 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:31:00.314345 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 637/1293] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:31:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:31:00.319616 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30e734-76f4-4e07-9f8c-dec421e826f7 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:31:00.321428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30e734-76f4-4e07-9f8c-dec421e826f7 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:31:00.321428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30e734-76f4-4e07-9f8c-dec421e826f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:31:00.337350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-3e30e734-76f4-4e07-9f8c-dec421e826f7 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:31:00.338538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3e30e734-76f4-4e07-9f8c-dec421e826f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:31:00.367380 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:31:00.404547 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 657/1294] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:31:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:31:00.428051 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0767e0-7477-4802-89ba-7b29975dc1e3 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:31:00.462606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:31:00.530880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0767e0-7477-4802-89ba-7b29975dc1e3 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:31:00.564363 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddae695-e907-44ef-b438-757c35757be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:31:00.564961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddae695-e907-44ef-b438-757c35757be1 None admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:31:00.565243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddae695-e907-44ef-b438-757c35757be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:31:00.565518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddae695-e907-44ef-b438-757c35757be1 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:31:00.566231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddae695-e907-44ef-b438-757c35757be1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:31:00.567655 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddae695-e907-44ef-b438-757c35757be1 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:31:00.661131 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 638/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 23 22:31:00 2022] POST /identity/v3/services => generated 246 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:31:02.722938 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0b117-d692-425e-ace2-cde9fba2177e None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:31:02.723463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0b117-d692-425e-ace2-cde9fba2177e None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:31:02.723463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0b117-d692-425e-ace2-cde9fba2177e None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:31:02.724870 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 658/1296] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:31:02.730115 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1e4da-4338-488d-aff1-5e82e92e1be8 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:31:02.730549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1e4da-4338-488d-aff1-5e82e92e1be8 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:31:02.730661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1e4da-4338-488d-aff1-5e82e92e1be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:31:02.747880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-a3d1e4da-4338-488d-aff1-5e82e92e1be8 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:31:02.749085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-a3d1e4da-4338-488d-aff1-5e82e92e1be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:31:02.821692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 639/1297] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:31:02 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:31:02.845700 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0b117-d692-425e-ace2-cde9fba2177e None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:31:02.947641 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0b117-d692-425e-ace2-cde9fba2177e None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:31:02.981076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d02703-681a-4a43-aece-3ff6537228e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:31:02.981860 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d02703-681a-4a43-aece-3ff6537228e4 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:31:02.982142 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d02703-681a-4a43-aece-3ff6537228e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:31:02.982416 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d02703-681a-4a43-aece-3ff6537228e4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:31:02.983060 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d02703-681a-4a43-aece-3ff6537228e4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:31:02.984408 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d02703-681a-4a43-aece-3ff6537228e4 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:31:02.995929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-94d02703-681a-4a43-aece-3ff6537228e4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 22:31:02.995929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:31:02.995929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:31:02.995929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:31:02.995929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:31:02.995929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:31:02.995929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:31:02.995929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:31:02.995929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 22:31:02.995929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:31:02.995929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:31:02.995929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:31:02.995929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:31:02.995929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:31:02.995929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:31:02.995929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:31:02.995929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:31:02.997084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:31:02.997084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:31:02.997084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:31:02.997084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:31:02.997084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:31:02.997084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:31:02.997084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:31:02.997084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:31:02.997084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:31:02.997084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:31:02.997084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:31:02.997084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:31:02.997084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:31:02.997084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:31:02.997084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:31:02.997084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:31:02.998099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:31:02.998099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:31:02.998099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:31:02.998099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:31:02.998099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:31:02.998099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:31:02.998099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:31:02.998099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:31:02.998099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:31:02.998099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:31:02.998099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:31:02.998099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:31:02.998099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 22:31:02.998099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:31:02.998099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 659/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 23 22:31:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:31:03.003276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1e4da-4338-488d-aff1-5e82e92e1be8 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:31:03.021713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:31:03.100434 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1e4da-4338-488d-aff1-5e82e92e1be8 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:31:03.133613 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f89e9-4f67-4653-86fb-415f5b077e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:31:03.134291 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f89e9-4f67-4653-86fb-415f5b077e93 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:31:03.134577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f89e9-4f67-4653-86fb-415f5b077e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:31:03.134853 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f89e9-4f67-4653-86fb-415f5b077e93 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:31:03.135553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f89e9-4f67-4653-86fb-415f5b077e93 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:31:03.135858 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f89e9-4f67-4653-86fb-415f5b077e93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:31:03.137259 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f89e9-4f67-4653-86fb-415f5b077e93 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:31:03.142969 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 640/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Jun 23 22:31:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:31:03.148918 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d02703-681a-4a43-aece-3ff6537228e4 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:31:03.180256 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:31:03.247289 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d02703-681a-4a43-aece-3ff6537228e4 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:31:03.280570 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab46c9f-cbc1-470d-9502-b5202bfd5f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:31:03.281248 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab46c9f-cbc1-470d-9502-b5202bfd5f7d None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:31:03.282114 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab46c9f-cbc1-470d-9502-b5202bfd5f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:31:03.282451 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab46c9f-cbc1-470d-9502-b5202bfd5f7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:31:03.283534 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab46c9f-cbc1-470d-9502-b5202bfd5f7d None admin] RBAC: Adding query filter params (interface=public, service_id=8341d4e135724365ada5bb5eeb126bf0, region_id=RegionOne) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:31:03.283845 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab46c9f-cbc1-470d-9502-b5202bfd5f7d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:31:03.285231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab46c9f-cbc1-470d-9502-b5202bfd5f7d None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:31:03.290680 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 660/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 23 22:31:03 2022] GET /identity/v3/endpoints?service_id=8341d4e135724365ada5bb5eeb126bf0&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:31:03.296281 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f89e9-4f67-4653-86fb-415f5b077e93 None admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:31:03.378803 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:31:03.397283 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f89e9-4f67-4653-86fb-415f5b077e93 None admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:31:03.431237 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0b86a-5d76-4527-8c5f-4c48c56dca4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:31:03.431898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0b86a-5d76-4527-8c5f-4c48c56dca4c None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:31:03.432183 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0b86a-5d76-4527-8c5f-4c48c56dca4c None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:31:03.432465 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0b86a-5d76-4527-8c5f-4c48c56dca4c None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:31:03.433199 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0b86a-5d76-4527-8c5f-4c48c56dca4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:31:03.434558 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0b86a-5d76-4527-8c5f-4c48c56dca4c None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:31:03.441911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 641/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 23 22:31:03 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:31:05.452893 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67758884-ffbc-4c9d-a9d8-1ccbefebb4c7 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:31:05.453534 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67758884-ffbc-4c9d-a9d8-1ccbefebb4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:31:05.453705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67758884-ffbc-4c9d-a9d8-1ccbefebb4c7 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:31:05.455361 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 661/1302] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:31:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:31:05.460762 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e311d77-5a5d-4e08-8996-af7db1ce98b3 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:31:05.461158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e311d77-5a5d-4e08-8996-af7db1ce98b3 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:31:05.461333 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e311d77-5a5d-4e08-8996-af7db1ce98b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:31:05.477734 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-6e311d77-5a5d-4e08-8996-af7db1ce98b3 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:31:05.478910 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-6e311d77-5a5d-4e08-8996-af7db1ce98b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:31:05.545120 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 642/1303] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:31:05 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:31:05.568801 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67758884-ffbc-4c9d-a9d8-1ccbefebb4c7 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:31:05.664182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:31:05.670750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67758884-ffbc-4c9d-a9d8-1ccbefebb4c7 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:31:05.704255 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21d426-89e3-49e5-ab57-f628db5eee99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:31:05.705082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21d426-89e3-49e5-ab57-f628db5eee99 None admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:31:05.705367 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21d426-89e3-49e5-ab57-f628db5eee99 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:31:05.705643 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21d426-89e3-49e5-ab57-f628db5eee99 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:31:05.706289 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21d426-89e3-49e5-ab57-f628db5eee99 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:31:05.707645 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21d426-89e3-49e5-ab57-f628db5eee99 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:31:05.720273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application [None req-bd21d426-89e3-49e5-ab57-f628db5eee99 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 22:31:05.720273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:31:05.720273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:31:05.720273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:31:05.720273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:31:05.720273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:31:05.720273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:31:05.720273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:31:05.720273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 22:31:05.720273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:31:05.720273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:31:05.720273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:31:05.720273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:31:05.720273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:31:05.720273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:31:05.720273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:31:05.720273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:31:05.721638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:31:05.721638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:31:05.721638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:31:05.721638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:31:05.721638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:31:05.721638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:31:05.721638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:31:05.721638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:31:05.721638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:31:05.721638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:31:05.721638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:31:05.721638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:31:05.721638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:31:05.721638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:31:05.721638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:31:05.721638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:31:05.722683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:31:05.722683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application with Lock( Jun 23 22:31:05.722683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:31:05.722683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:31:05.722683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:31:05.722683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:31:05.722683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:31:05.722683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:31:05.722683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:31:05.722683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:31:05.722683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:31:05.722683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:31:05.722683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 22:31:05.722683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: ERROR keystone.server.flask.application Jun 23 22:31:05.722683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 662/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 23 22:31:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:31:05.727568 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e311d77-5a5d-4e08-8996-af7db1ce98b3 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:31:05.804899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:31:05.827759 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e311d77-5a5d-4e08-8996-af7db1ce98b3 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:31:05.861207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2bfc0-a737-4f8f-b9a1-787f04927da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:31:05.861861 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2bfc0-a737-4f8f-b9a1-787f04927da0 None admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:31:05.862147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2bfc0-a737-4f8f-b9a1-787f04927da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:31:05.862425 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2bfc0-a737-4f8f-b9a1-787f04927da0 None admin] PATH_INFO: `/v3/services` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:31:05.863123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2bfc0-a737-4f8f-b9a1-787f04927da0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:31:05.863433 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2bfc0-a737-4f8f-b9a1-787f04927da0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:31:05.864859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2bfc0-a737-4f8f-b9a1-787f04927da0 None admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:31:05.869881 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 643/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Jun 23 22:31:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:31:05.875740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21d426-89e3-49e5-ab57-f628db5eee99 None admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:31:05.975280 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21d426-89e3-49e5-ab57-f628db5eee99 None admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:31:06.007355 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a181311-0001-4e11-a912-0fc97c12ce75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:31:06.007994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a181311-0001-4e11-a912-0fc97c12ce75 None admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:31:06.008278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a181311-0001-4e11-a912-0fc97c12ce75 None admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:31:06.008556 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a181311-0001-4e11-a912-0fc97c12ce75 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:31:06.009325 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a181311-0001-4e11-a912-0fc97c12ce75 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:31:06.010681 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a181311-0001-4e11-a912-0fc97c12ce75 None admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:31:06.079301 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 663/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 23 22:31:05 2022] POST /identity/v3/endpoints => generated 331 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:31:15.866927 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=30 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:31:16.073764 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:41.040437 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ec369-8264-47a3-bf83-1eb2676fa3c7 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:41.041041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ec369-8264-47a3-bf83-1eb2676fa3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:41.041544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ec369-8264-47a3-bf83-1eb2676fa3c7 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:41.049344 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2bb6f-cfa8-4a48-aab4-b2be48b1b45a None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:41.049825 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2bb6f-cfa8-4a48-aab4-b2be48b1b45a None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:41.049923 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2bb6f-cfa8-4a48-aab4-b2be48b1b45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:41.053573 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 644/1307] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:36:41 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:36:41.147978 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-4be2bb6f-cfa8-4a48-aab4-b2be48b1b45a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:41.149256 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-4be2bb6f-cfa8-4a48-aab4-b2be48b1b45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:41.292974 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 664/1308] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:36:41.324184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17901a-4ec2-4a97-a3a3-d10d05b37ce7 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:41.324480 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17901a-4ec2-4a97-a3a3-d10d05b37ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:41.324867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17901a-4ec2-4a97-a3a3-d10d05b37ce7 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:41.326416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 645/1309] 10.222.0.33 () {54 vars in 964 bytes} [Thu Jun 23 22:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:36:41.332634 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3fa70-77ba-44a1-aa3a-59201338f231 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:41.333121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3fa70-77ba-44a1-aa3a-59201338f231 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:41.333413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3fa70-77ba-44a1-aa3a-59201338f231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:41.415866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-a7d3fa70-77ba-44a1-aa3a-59201338f231 None None] MFA Rules not processed for user `ec7412a9a3dc46369fe7b4771e3437de`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:41.417149 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a7d3fa70-77ba-44a1-aa3a-59201338f231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:41.444883 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:41.524401 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 665/1310] 10.222.0.33 () {58 vars in 1061 bytes} [Thu Jun 23 22:36:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:41.533854 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17901a-4ec2-4a97-a3a3-d10d05b37ce7 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:41.535608 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-5e17901a-4ec2-4a97-a3a3-d10d05b37ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:41.654248 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17901a-4ec2-4a97-a3a3-d10d05b37ce7 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:41.733668 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184b8ab-ffa8-4e5f-9d76-ded3917d53b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:41.734318 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184b8ab-ffa8-4e5f-9d76-ded3917d53b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:41.734602 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184b8ab-ffa8-4e5f-9d76-ded3917d53b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:41.734874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184b8ab-ffa8-4e5f-9d76-ded3917d53b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:41.736492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-9184b8ab-ffa8-4e5f-9d76-ded3917d53b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:41.792074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:41.848184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184b8ab-ffa8-4e5f-9d76-ded3917d53b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:41.849698 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184b8ab-ffa8-4e5f-9d76-ded3917d53b3 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:41.869583 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 646/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:44.224070 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6385909-ddf9-4e41-938b-944dc43b38f2 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:44.224497 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6385909-ddf9-4e41-938b-944dc43b38f2 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:44.224692 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6385909-ddf9-4e41-938b-944dc43b38f2 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:44.226312 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 666/1312] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:36:44.232043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c32f97-1872-45f1-ae33-11223ed0bbb8 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:44.232514 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c32f97-1872-45f1-ae33-11223ed0bbb8 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:44.233199 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c32f97-1872-45f1-ae33-11223ed0bbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:44.305916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-f7c32f97-1872-45f1-ae33-11223ed0bbb8 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:44.307120 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-f7c32f97-1872-45f1-ae33-11223ed0bbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:44.424637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 647/1313] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:44.461968 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6385909-ddf9-4e41-938b-944dc43b38f2 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:44.463544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-f6385909-ddf9-4e41-938b-944dc43b38f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:44.580869 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6385909-ddf9-4e41-938b-944dc43b38f2 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:44.659557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:44.662458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1a3e8-73d7-45ce-a858-ceaf18ad45b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:44.663198 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1a3e8-73d7-45ce-a858-ceaf18ad45b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:44.663556 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1a3e8-73d7-45ce-a858-ceaf18ad45b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:44.663905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1a3e8-73d7-45ce-a858-ceaf18ad45b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:44.665862 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-80f1a3e8-73d7-45ce-a858-ceaf18ad45b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:44.789868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1a3e8-73d7-45ce-a858-ceaf18ad45b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:44.791472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1a3e8-73d7-45ce-a858-ceaf18ad45b6 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:44.812600 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 667/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:47.447583 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2faddcf-63f6-4bef-b4cc-c761d87cc2e8 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:47.448105 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2faddcf-63f6-4bef-b4cc-c761d87cc2e8 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:47.448225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2faddcf-63f6-4bef-b4cc-c761d87cc2e8 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:47.449587 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 648/1315] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:36:47.454956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ca4f9-67bf-4f4a-82ee-bd25799e3650 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:47.455346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ca4f9-67bf-4f4a-82ee-bd25799e3650 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:47.455513 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ca4f9-67bf-4f4a-82ee-bd25799e3650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:47.519695 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-264ca4f9-67bf-4f4a-82ee-bd25799e3650 None None] MFA Rules not processed for user `65e4eacde0544043820a4928b1ac4083`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:47.520913 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-264ca4f9-67bf-4f4a-82ee-bd25799e3650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:47.572266 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:47.615920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 668/1316] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:36:47 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:47.656169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2faddcf-63f6-4bef-b4cc-c761d87cc2e8 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:47.657750 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-f2faddcf-63f6-4bef-b4cc-c761d87cc2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:47.667304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:47.760926 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2faddcf-63f6-4bef-b4cc-c761d87cc2e8 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:47.834401 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2294f9-ef33-4273-9460-cdc768f13021 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:47.834979 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2294f9-ef33-4273-9460-cdc768f13021 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:47.835268 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2294f9-ef33-4273-9460-cdc768f13021 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:47.835549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2294f9-ef33-4273-9460-cdc768f13021 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:47.837226 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ff2294f9-ef33-4273-9460-cdc768f13021 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:47.897806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:47.944134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2294f9-ef33-4273-9460-cdc768f13021 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:47.945597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2294f9-ef33-4273-9460-cdc768f13021 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:47.963057 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 649/1317] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:47 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:36:50.718861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b7f74-a81f-4c89-9567-283bb2771da0 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:50.719374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b7f74-a81f-4c89-9567-283bb2771da0 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:50.719374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b7f74-a81f-4c89-9567-283bb2771da0 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:50.720751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 669/1318] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:36:50.725820 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794edfdf-be84-419f-bf53-6e0ecdbc0fcb None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:50.726259 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794edfdf-be84-419f-bf53-6e0ecdbc0fcb None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:50.726381 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794edfdf-be84-419f-bf53-6e0ecdbc0fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:50.789860 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-794edfdf-be84-419f-bf53-6e0ecdbc0fcb None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:50.791062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-794edfdf-be84-419f-bf53-6e0ecdbc0fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:50.885126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 650/1319] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:36:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:50.939064 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93579d55-e31c-4577-a2aa-d315c7e1ba13 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:50.939350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93579d55-e31c-4577-a2aa-d315c7e1ba13 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:50.939628 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93579d55-e31c-4577-a2aa-d315c7e1ba13 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:50.941064 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 670/1320] 10.222.0.33 () {54 vars in 964 bytes} [Thu Jun 23 22:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:36:50.947031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0fde8-2eaa-42bf-bcce-40c4d768d410 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:50.947323 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0fde8-2eaa-42bf-bcce-40c4d768d410 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:50.947602 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0fde8-2eaa-42bf-bcce-40c4d768d410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:50.968212 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:51.021180 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-69a0fde8-2eaa-42bf-bcce-40c4d768d410 None None] MFA Rules not processed for user `ec7412a9a3dc46369fe7b4771e3437de`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:51.022376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-69a0fde8-2eaa-42bf-bcce-40c4d768d410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:51.097934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 651/1321] 10.222.0.33 () {58 vars in 1061 bytes} [Thu Jun 23 22:36:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:51.106829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93579d55-e31c-4577-a2aa-d315c7e1ba13 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:51.108683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-93579d55-e31c-4577-a2aa-d315c7e1ba13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:51.208456 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93579d55-e31c-4577-a2aa-d315c7e1ba13 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:51.281033 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f9307-acf2-4260-ab8a-e16ca0ad62d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:51.281622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f9307-acf2-4260-ab8a-e16ca0ad62d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:51.281907 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f9307-acf2-4260-ab8a-e16ca0ad62d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:51.282183 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f9307-acf2-4260-ab8a-e16ca0ad62d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:51.283829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-969f9307-acf2-4260-ab8a-e16ca0ad62d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:51.292300 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:51.392538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f9307-acf2-4260-ab8a-e16ca0ad62d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:51.394327 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f9307-acf2-4260-ab8a-e16ca0ad62d7 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:51.412298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 671/1322] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:54.795347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79303b3a-6516-4ba7-8d4e-84885f4796cb None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:54.795877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79303b3a-6516-4ba7-8d4e-84885f4796cb None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:54.795977 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79303b3a-6516-4ba7-8d4e-84885f4796cb None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:54.797306 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 652/1323] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:36:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:36:54.802541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb7742-550f-4987-8dbb-fb570b0dabd2 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:54.802989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb7742-550f-4987-8dbb-fb570b0dabd2 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:54.803234 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb7742-550f-4987-8dbb-fb570b0dabd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:54.868411 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-2bcb7742-550f-4987-8dbb-fb570b0dabd2 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:54.869912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-2bcb7742-550f-4987-8dbb-fb570b0dabd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:54.927698 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:54.967916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 672/1324] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:55.005075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79303b3a-6516-4ba7-8d4e-84885f4796cb None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:55.006594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-79303b3a-6516-4ba7-8d4e-84885f4796cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:55.119244 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79303b3a-6516-4ba7-8d4e-84885f4796cb None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:55.132370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:55.197311 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74714dfa-7e65-41df-909b-2f9497a719f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:55.197960 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74714dfa-7e65-41df-909b-2f9497a719f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:55.198243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74714dfa-7e65-41df-909b-2f9497a719f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:55.198516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74714dfa-7e65-41df-909b-2f9497a719f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:55.200139 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-74714dfa-7e65-41df-909b-2f9497a719f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:55.313377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74714dfa-7e65-41df-909b-2f9497a719f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:55.314812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74714dfa-7e65-41df-909b-2f9497a719f2 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:55.333601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 653/1325] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:57.436727 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3af678-6730-472b-9e5c-2a7c73cd1149 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:57.437282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3af678-6730-472b-9e5c-2a7c73cd1149 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:57.437282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3af678-6730-472b-9e5c-2a7c73cd1149 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:57.438691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 673/1326] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:36:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:36:57.444117 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2050ff7-f788-4e2c-92b5-d19f58600833 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:57.444556 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2050ff7-f788-4e2c-92b5-d19f58600833 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:57.444790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2050ff7-f788-4e2c-92b5-d19f58600833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:57.493096 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:57.511832 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-a2050ff7-f788-4e2c-92b5-d19f58600833 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:57.513685 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-a2050ff7-f788-4e2c-92b5-d19f58600833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:57.611979 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 654/1327] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:36:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:57.649884 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3af678-6730-472b-9e5c-2a7c73cd1149 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:57.653810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-2c3af678-6730-472b-9e5c-2a7c73cd1149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:57.752785 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3af678-6730-472b-9e5c-2a7c73cd1149 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:57.826272 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167ec15-d5df-47c4-bf4d-5f92cff0c7f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:57.826840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167ec15-d5df-47c4-bf4d-5f92cff0c7f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:57.827124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167ec15-d5df-47c4-bf4d-5f92cff0c7f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:57.827401 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167ec15-d5df-47c4-bf4d-5f92cff0c7f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:57.829049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3167ec15-d5df-47c4-bf4d-5f92cff0c7f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:57.845834 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:57.944992 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167ec15-d5df-47c4-bf4d-5f92cff0c7f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:57.946451 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167ec15-d5df-47c4-bf4d-5f92cff0c7f4 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:57.965619 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 674/1328] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:00.058128 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04cd67-d4c3-4eb9-b1f0-0d99d0f23523 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:00.058706 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04cd67-d4c3-4eb9-b1f0-0d99d0f23523 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:00.058706 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04cd67-d4c3-4eb9-b1f0-0d99d0f23523 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:00.060239 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 655/1329] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:00.066442 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686f689-76f6-483a-8c88-349731ce0243 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:00.067008 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686f689-76f6-483a-8c88-349731ce0243 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:00.067008 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686f689-76f6-483a-8c88-349731ce0243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:00.133372 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-6686f689-76f6-483a-8c88-349731ce0243 None None] MFA Rules not processed for user `65e4eacde0544043820a4928b1ac4083`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:00.134604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6686f689-76f6-483a-8c88-349731ce0243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:00.199038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:00.231318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 675/1330] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:37:00 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:00.269571 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04cd67-d4c3-4eb9-b1f0-0d99d0f23523 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:00.271124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3e04cd67-d4c3-4eb9-b1f0-0d99d0f23523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:00.381610 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04cd67-d4c3-4eb9-b1f0-0d99d0f23523 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:00.401555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:00.456648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7b261-0cc4-4d4e-946d-344da6721e7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:00.457305 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7b261-0cc4-4d4e-946d-344da6721e7a service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:00.457589 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7b261-0cc4-4d4e-946d-344da6721e7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:00.457866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7b261-0cc4-4d4e-946d-344da6721e7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:00.459501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-80f7b261-0cc4-4d4e-946d-344da6721e7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:00.571642 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f7b261-0cc4-4d4e-946d-344da6721e7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:00.572929 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f7b261-0cc4-4d4e-946d-344da6721e7a service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:00.592773 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 656/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:00 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:03.094123 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d8a24-4bfd-47f4-b02a-effc0d41fa8e None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:03.094678 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d8a24-4bfd-47f4-b02a-effc0d41fa8e None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:03.094678 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d8a24-4bfd-47f4-b02a-effc0d41fa8e None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:03.096058 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 676/1332] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:03.101305 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a82c5-315c-4d70-86c9-70bbc13377bc None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:03.101685 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a82c5-315c-4d70-86c9-70bbc13377bc None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:03.101871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a82c5-315c-4d70-86c9-70bbc13377bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:03.163441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:03.166348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-092a82c5-315c-4d70-86c9-70bbc13377bc None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:03.167549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-092a82c5-315c-4d70-86c9-70bbc13377bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:03.262457 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 657/1333] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:03.299076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d8a24-4bfd-47f4-b02a-effc0d41fa8e None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:03.300642 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ab0d8a24-4bfd-47f4-b02a-effc0d41fa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:03.404020 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d8a24-4bfd-47f4-b02a-effc0d41fa8e None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:03.476785 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440fca86-0bb6-4399-beaa-a69d38560ff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:03.477429 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fca86-0bb6-4399-beaa-a69d38560ff9 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:03.477709 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fca86-0bb6-4399-beaa-a69d38560ff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:03.477979 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fca86-0bb6-4399-beaa-a69d38560ff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:03.479591 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-440fca86-0bb6-4399-beaa-a69d38560ff9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:03.506104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:03.588139 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440fca86-0bb6-4399-beaa-a69d38560ff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:03.589570 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440fca86-0bb6-4399-beaa-a69d38560ff9 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:03.607644 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 677/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:05.687318 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8470f2-1d9c-4beb-aa31-98e32eabb7da None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:05.687843 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8470f2-1d9c-4beb-aa31-98e32eabb7da None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:05.687843 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8470f2-1d9c-4beb-aa31-98e32eabb7da None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:05.689190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 658/1335] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:05.694265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39e9eb-3703-4c55-9041-aad4686de79b None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:05.694663 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39e9eb-3703-4c55-9041-aad4686de79b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:05.694819 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39e9eb-3703-4c55-9041-aad4686de79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:05.758761 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-2c39e9eb-3703-4c55-9041-aad4686de79b None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:05.759949 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-2c39e9eb-3703-4c55-9041-aad4686de79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:05.831894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:05.853822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 678/1336] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:05.893759 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8470f2-1d9c-4beb-aa31-98e32eabb7da None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:05.895320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-7f8470f2-1d9c-4beb-aa31-98e32eabb7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:05.999412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8470f2-1d9c-4beb-aa31-98e32eabb7da None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:06.027677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:06.072158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc88b27-5d32-4329-9428-775ee7555483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:06.072831 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc88b27-5d32-4329-9428-775ee7555483 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:06.073117 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc88b27-5d32-4329-9428-775ee7555483 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:06.073392 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc88b27-5d32-4329-9428-775ee7555483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:06.075552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-dfc88b27-5d32-4329-9428-775ee7555483 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:06.181300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc88b27-5d32-4329-9428-775ee7555483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:06.182729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc88b27-5d32-4329-9428-775ee7555483 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:06.201326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 659/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:08.879578 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e817d7a-19e4-49cc-b506-157a37aecae2 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:08.880127 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e817d7a-19e4-49cc-b506-157a37aecae2 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:08.880127 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e817d7a-19e4-49cc-b506-157a37aecae2 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:08.881547 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 679/1338] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:08.886621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6740a-78b0-428d-9bec-82e726af18b3 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:08.887036 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6740a-78b0-428d-9bec-82e726af18b3 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:08.887179 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6740a-78b0-428d-9bec-82e726af18b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:08.950267 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-74b6740a-78b0-428d-9bec-82e726af18b3 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:08.951454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-74b6740a-78b0-428d-9bec-82e726af18b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:08.959455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:09.047104 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 660/1339] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:37:08 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:09.083024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e817d7a-19e4-49cc-b506-157a37aecae2 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:09.084553 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3e817d7a-19e4-49cc-b506-157a37aecae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:09.185757 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e817d7a-19e4-49cc-b506-157a37aecae2 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:09.256474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2866db7-20fc-4f0e-8df0-c00c63114e41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:09.257141 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2866db7-20fc-4f0e-8df0-c00c63114e41 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:09.257426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2866db7-20fc-4f0e-8df0-c00c63114e41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:09.257709 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2866db7-20fc-4f0e-8df0-c00c63114e41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:09.259316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-c2866db7-20fc-4f0e-8df0-c00c63114e41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:09.295177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:09.367885 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2866db7-20fc-4f0e-8df0-c00c63114e41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:09.369256 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2866db7-20fc-4f0e-8df0-c00c63114e41 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:09.388954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 680/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:13.665052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32449da7-3975-4cd1-8530-a216d781b955 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:13.665576 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32449da7-3975-4cd1-8530-a216d781b955 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:13.665676 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32449da7-3975-4cd1-8530-a216d781b955 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:13.667076 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 661/1341] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:13.673410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d71507-8b68-4e23-9fc2-93febb194431 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:13.673845 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d71507-8b68-4e23-9fc2-93febb194431 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:13.674139 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d71507-8b68-4e23-9fc2-93febb194431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:13.758091 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-b9d71507-8b68-4e23-9fc2-93febb194431 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:13.759295 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-b9d71507-8b68-4e23-9fc2-93febb194431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:13.852625 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:13.866028 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 681/1342] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:13.906501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32449da7-3975-4cd1-8530-a216d781b955 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:13.908769 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-32449da7-3975-4cd1-8530-a216d781b955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:14.006872 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32449da7-3975-4cd1-8530-a216d781b955 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:14.010691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a4fdf-46a0-4284-94d8-21d05ae71c6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:14.011343 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a4fdf-46a0-4284-94d8-21d05ae71c6f service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:14.011631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a4fdf-46a0-4284-94d8-21d05ae71c6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:14.011911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a4fdf-46a0-4284-94d8-21d05ae71c6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:14.021393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a4fdf-46a0-4284-94d8-21d05ae71c6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:14.022850 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a4fdf-46a0-4284-94d8-21d05ae71c6f service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:14.032873 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 662/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:16.969773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac70ff-0faf-4199-9c5b-62d45194a900 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:16.970314 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac70ff-0faf-4199-9c5b-62d45194a900 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:16.970314 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac70ff-0faf-4199-9c5b-62d45194a900 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:16.971717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 682/1344] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:16.977364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428db4d1-d064-4cd7-adf2-7e17000bd95c None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:16.977849 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428db4d1-d064-4cd7-adf2-7e17000bd95c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:16.977942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428db4d1-d064-4cd7-adf2-7e17000bd95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:17.008357 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-428db4d1-d064-4cd7-adf2-7e17000bd95c None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:17.009609 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-428db4d1-d064-4cd7-adf2-7e17000bd95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:17.042119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:17.092320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 663/1345] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:37:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:17.130771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac70ff-0faf-4199-9c5b-62d45194a900 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:17.142493 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac70ff-0faf-4199-9c5b-62d45194a900 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:17.146189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841eb2e4-db72-44b5-bebc-4db567302a96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:17.146824 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841eb2e4-db72-44b5-bebc-4db567302a96 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:17.147115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841eb2e4-db72-44b5-bebc-4db567302a96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:17.147395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841eb2e4-db72-44b5-bebc-4db567302a96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:17.156598 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841eb2e4-db72-44b5-bebc-4db567302a96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:17.158026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841eb2e4-db72-44b5-bebc-4db567302a96 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:17.167616 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 683/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:21.492817 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63197334-be62-4851-9c98-6b98c1697151 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:21.493334 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63197334-be62-4851-9c98-6b98c1697151 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:21.493452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63197334-be62-4851-9c98-6b98c1697151 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:21.494796 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 664/1347] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:21.500318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144169b-6e6b-4ae2-8384-46f9562913ea None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:21.500750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144169b-6e6b-4ae2-8384-46f9562913ea None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:21.500902 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144169b-6e6b-4ae2-8384-46f9562913ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:21.518710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-b144169b-6e6b-4ae2-8384-46f9562913ea None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:21.519912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-b144169b-6e6b-4ae2-8384-46f9562913ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:21.528456 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 684/1348] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:21.566009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63197334-be62-4851-9c98-6b98c1697151 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:21.577102 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63197334-be62-4851-9c98-6b98c1697151 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:21.580651 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318e2e6-1a37-4702-a9e8-2292bfcb3802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:21.581295 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318e2e6-1a37-4702-a9e8-2292bfcb3802 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:21.581580 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318e2e6-1a37-4702-a9e8-2292bfcb3802 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:21.581859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318e2e6-1a37-4702-a9e8-2292bfcb3802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:21.591700 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318e2e6-1a37-4702-a9e8-2292bfcb3802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:21.593174 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318e2e6-1a37-4702-a9e8-2292bfcb3802 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:21.602824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 665/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:26.578983 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18066a-06f6-484c-8e11-c466cf995210 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:26.580094 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18066a-06f6-484c-8e11-c466cf995210 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:26.580789 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18066a-06f6-484c-8e11-c466cf995210 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:26.582653 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 685/1350] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 22:37:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:26.588766 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7f078-9d37-44d6-9f95-b8cdc528ad89 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:26.589214 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7f078-9d37-44d6-9f95-b8cdc528ad89 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:26.589337 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7f078-9d37-44d6-9f95-b8cdc528ad89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:26.700262 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-20a7f078-9d37-44d6-9f95-b8cdc528ad89 None None] MFA Rules not processed for user `d3ac44b3f0224e56a15993a2f29c1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:26.701945 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-20a7f078-9d37-44d6-9f95-b8cdc528ad89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:26.789784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 666/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 22:37:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:31.036870 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abcafa-67fe-4ac1-a0e0-02b29103cc4f None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:31.037720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abcafa-67fe-4ac1-a0e0-02b29103cc4f None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:31.038007 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abcafa-67fe-4ac1-a0e0-02b29103cc4f None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:31.039569 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 686/1352] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 22:37:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:31.045405 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b618b5d-6a66-405b-868a-d101c4fba3b6 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:31.045844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b618b5d-6a66-405b-868a-d101c4fba3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:31.045948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b618b5d-6a66-405b-868a-d101c4fba3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:31.065312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-8b618b5d-6a66-405b-868a-d101c4fba3b6 None None] MFA Rules not processed for user `d3ac44b3f0224e56a15993a2f29c1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:31.066713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-8b618b5d-6a66-405b-868a-d101c4fba3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:31.075865 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 667/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 22:37:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:31.513521 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=13 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:32.478312 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0276b0-a2ec-4b84-9d11-284cc8e433a3 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:32.478956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0276b0-a2ec-4b84-9d11-284cc8e433a3 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:32.478956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0276b0-a2ec-4b84-9d11-284cc8e433a3 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:32.481136 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 687/1354] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 22:37:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:32.486380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256029b0-3f09-468d-bfe9-0fbd0e40cc57 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:32.486855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256029b0-3f09-468d-bfe9-0fbd0e40cc57 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:32.487069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256029b0-3f09-468d-bfe9-0fbd0e40cc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:32.506065 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-256029b0-3f09-468d-bfe9-0fbd0e40cc57 None None] MFA Rules not processed for user `d3ac44b3f0224e56a15993a2f29c1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:32.507462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-256029b0-3f09-468d-bfe9-0fbd0e40cc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:32.516508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 668/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 22:37:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:37:36.420037 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a782b0-1347-442e-bdcc-e050923857c0 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:36.420546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a782b0-1347-442e-bdcc-e050923857c0 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:36.420546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a782b0-1347-442e-bdcc-e050923857c0 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:36.422445 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 688/1356] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:36.428039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff0a47-2847-4921-9885-b8d9c6660c38 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:36.428865 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff0a47-2847-4921-9885-b8d9c6660c38 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:36.429371 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff0a47-2847-4921-9885-b8d9c6660c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:36.453876 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-e3ff0a47-2847-4921-9885-b8d9c6660c38 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:36.455340 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e3ff0a47-2847-4921-9885-b8d9c6660c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:36.467677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 669/1357] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:37:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:36.524760 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a2016-e59d-4882-acbf-c08eb3d6088b None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:36.525666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a2016-e59d-4882-acbf-c08eb3d6088b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:36.526353 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a2016-e59d-4882-acbf-c08eb3d6088b None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:36.529121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 689/1358] 10.222.0.33 () {54 vars in 961 bytes} [Thu Jun 23 22:37:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:36.537572 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f833b3-6bc4-4044-a124-5e93d8ecffd7 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:36.538644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f833b3-6bc4-4044-a124-5e93d8ecffd7 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:36.539080 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f833b3-6bc4-4044-a124-5e93d8ecffd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:36.586896 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:36.594962 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-12f833b3-6bc4-4044-a124-5e93d8ecffd7 None None] MFA Rules not processed for user `159cc8befc3a4d57bbd313832084db62`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:36.596144 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-12f833b3-6bc4-4044-a124-5e93d8ecffd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:36.684261 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 670/1359] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Jun 23 22:37:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:36.692660 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a2016-e59d-4882-acbf-c08eb3d6088b None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:36.708835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a2016-e59d-4882-acbf-c08eb3d6088b None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:36.715082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7790c29-5b82-4922-8fe2-aaa6410d7370 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:36.715746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7790c29-5b82-4922-8fe2-aaa6410d7370 service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:36.716033 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7790c29-5b82-4922-8fe2-aaa6410d7370 service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:36.716316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7790c29-5b82-4922-8fe2-aaa6410d7370 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:36.726220 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7790c29-5b82-4922-8fe2-aaa6410d7370 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:36.728196 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7790c29-5b82-4922-8fe2-aaa6410d7370 service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:36.738815 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 690/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:37:38.995866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175693d4-527d-4dad-9040-b5af567d2363 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:38.996424 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175693d4-527d-4dad-9040-b5af567d2363 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:38.996424 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175693d4-527d-4dad-9040-b5af567d2363 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:38.998089 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 671/1361] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:37:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:39.003373 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97ba94-ca79-4746-840c-c836dbc97511 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:39.003856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97ba94-ca79-4746-840c-c836dbc97511 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:39.003939 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97ba94-ca79-4746-840c-c836dbc97511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:39.022988 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-9b97ba94-ca79-4746-840c-c836dbc97511 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:39.024180 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-9b97ba94-ca79-4746-840c-c836dbc97511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:39.036962 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 691/1362] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:39.079143 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175693d4-527d-4dad-9040-b5af567d2363 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:39.096355 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175693d4-527d-4dad-9040-b5af567d2363 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:39.103230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e0f6c-4d43-479f-b468-670096ba3a27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:39.104224 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e0f6c-4d43-479f-b468-670096ba3a27 service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:39.105464 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e0f6c-4d43-479f-b468-670096ba3a27 service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:39.105955 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e0f6c-4d43-479f-b468-670096ba3a27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:39.116119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e0f6c-4d43-479f-b468-670096ba3a27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:39.118252 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e0f6c-4d43-479f-b468-670096ba3a27 service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:39.131820 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 672/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:41.692755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67674b-f1f8-4da1-8a52-3739c882be74 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:41.693267 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67674b-f1f8-4da1-8a52-3739c882be74 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:41.693267 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67674b-f1f8-4da1-8a52-3739c882be74 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:41.694603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 692/1364] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:41.700319 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120edb23-198d-4399-8acd-310c92d4ea79 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:41.700779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120edb23-198d-4399-8acd-310c92d4ea79 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:41.700935 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120edb23-198d-4399-8acd-310c92d4ea79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:41.719351 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-120edb23-198d-4399-8acd-310c92d4ea79 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:41.720528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-120edb23-198d-4399-8acd-310c92d4ea79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:41.728171 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 673/1365] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:41.775159 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ec3df-1236-489e-b31b-fd601c41dd84 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:41.775390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ec3df-1236-489e-b31b-fd601c41dd84 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:41.775674 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ec3df-1236-489e-b31b-fd601c41dd84 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:41.777715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 693/1366] 10.222.0.33 () {54 vars in 961 bytes} [Thu Jun 23 22:37:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:41.783278 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740f753-4eb7-4699-a962-14559b15ad1e None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:41.784343 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740f753-4eb7-4699-a962-14559b15ad1e None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:41.784343 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740f753-4eb7-4699-a962-14559b15ad1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:41.803768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-f740f753-4eb7-4699-a962-14559b15ad1e None None] MFA Rules not processed for user `159cc8befc3a4d57bbd313832084db62`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:41.804983 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-f740f753-4eb7-4699-a962-14559b15ad1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:41.814084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 674/1367] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Jun 23 22:37:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:41.821707 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ec3df-1236-489e-b31b-fd601c41dd84 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:41.836367 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ec3df-1236-489e-b31b-fd601c41dd84 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:41.842597 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03f2ac-ca44-4a77-9e70-4f1dfee6b9d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:41.843236 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03f2ac-ca44-4a77-9e70-4f1dfee6b9d8 service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:41.843523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03f2ac-ca44-4a77-9e70-4f1dfee6b9d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:41.843810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03f2ac-ca44-4a77-9e70-4f1dfee6b9d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:41.852233 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03f2ac-ca44-4a77-9e70-4f1dfee6b9d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:41.853869 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03f2ac-ca44-4a77-9e70-4f1dfee6b9d8 service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:41.864001 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 694/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:44.104407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26b35c-983b-4665-8e8e-59cb0b27f563 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:44.105026 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26b35c-983b-4665-8e8e-59cb0b27f563 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:44.105026 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26b35c-983b-4665-8e8e-59cb0b27f563 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:44.106357 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 675/1369] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:44.111632 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b951fc-3891-4bdd-b94b-ed79e3e3917e None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:44.112036 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b951fc-3891-4bdd-b94b-ed79e3e3917e None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:44.112198 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b951fc-3891-4bdd-b94b-ed79e3e3917e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:44.130487 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-38b951fc-3891-4bdd-b94b-ed79e3e3917e None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:44.131669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-38b951fc-3891-4bdd-b94b-ed79e3e3917e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:44.140285 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 695/1370] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:44.184194 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26b35c-983b-4665-8e8e-59cb0b27f563 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:44.199698 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26b35c-983b-4665-8e8e-59cb0b27f563 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:44.205307 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f64824-e303-438b-84de-5b1d752c5ba1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:44.206102 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f64824-e303-438b-84de-5b1d752c5ba1 service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:44.206479 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f64824-e303-438b-84de-5b1d752c5ba1 service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:44.206829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f64824-e303-438b-84de-5b1d752c5ba1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:44.215836 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f64824-e303-438b-84de-5b1d752c5ba1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:44.217487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f64824-e303-438b-84de-5b1d752c5ba1 service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:44.227382 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 676/1371] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:44.687519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf129e3-9224-4e12-86a0-972070267c73 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:44.688259 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf129e3-9224-4e12-86a0-972070267c73 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:44.688825 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf129e3-9224-4e12-86a0-972070267c73 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:44.690518 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 696/1372] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:37:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:44.696198 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516979c5-6753-4474-8c68-c0628bd61e45 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:44.696650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516979c5-6753-4474-8c68-c0628bd61e45 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:44.697043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516979c5-6753-4474-8c68-c0628bd61e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:44.716027 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-516979c5-6753-4474-8c68-c0628bd61e45 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:44.717414 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-516979c5-6753-4474-8c68-c0628bd61e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:44.728622 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 677/1373] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:45.088296 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffcdce-2215-4fff-86e5-662bbd96ac23 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:45.089352 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffcdce-2215-4fff-86e5-662bbd96ac23 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:45.089731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffcdce-2215-4fff-86e5-662bbd96ac23 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:45.091405 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 697/1374] 10.222.0.33 () {54 vars in 963 bytes} [Thu Jun 23 22:37:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:45.096693 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9e442-aac5-4449-b62a-a9b51b227deb None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:45.097112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9e442-aac5-4449-b62a-a9b51b227deb None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:45.097510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9e442-aac5-4449-b62a-a9b51b227deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:45.148014 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-b9b9e442-aac5-4449-b62a-a9b51b227deb None None] MFA Rules not processed for user `5d1911bfac984e058b0b11bb6400e782`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:45.149404 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-b9b9e442-aac5-4449-b62a-a9b51b227deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:45.213940 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 678/1375] 10.222.0.33 () {58 vars in 1060 bytes} [Thu Jun 23 22:37:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:45.221822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffcdce-2215-4fff-86e5-662bbd96ac23 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:45.233692 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffcdce-2215-4fff-86e5-662bbd96ac23 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:45.237421 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394491f-195d-4feb-8be7-47a6dc53401a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:45.238068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394491f-195d-4feb-8be7-47a6dc53401a service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:45.238359 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394491f-195d-4feb-8be7-47a6dc53401a service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:45.238641 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394491f-195d-4feb-8be7-47a6dc53401a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:45.248509 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394491f-195d-4feb-8be7-47a6dc53401a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:45.249966 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394491f-195d-4feb-8be7-47a6dc53401a service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:45.260003 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 698/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:46.461842 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19439d3f-42f8-4548-9e9c-c81a96f9c7a8 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:46.462433 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19439d3f-42f8-4548-9e9c-c81a96f9c7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:46.462433 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19439d3f-42f8-4548-9e9c-c81a96f9c7a8 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:46.463910 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 679/1377] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:46.469403 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03916a5c-7a82-4bce-a770-2392c594bbe7 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:46.469804 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03916a5c-7a82-4bce-a770-2392c594bbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:46.469982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03916a5c-7a82-4bce-a770-2392c594bbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:46.487738 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-03916a5c-7a82-4bce-a770-2392c594bbe7 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:46.488983 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-03916a5c-7a82-4bce-a770-2392c594bbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:46.497739 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 699/1378] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:46.543192 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19439d3f-42f8-4548-9e9c-c81a96f9c7a8 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:46.558320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19439d3f-42f8-4548-9e9c-c81a96f9c7a8 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:46.563872 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f213e0-6cfe-427c-b4f2-2c503efa1d3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:46.564570 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f213e0-6cfe-427c-b4f2-2c503efa1d3e service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:46.564881 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f213e0-6cfe-427c-b4f2-2c503efa1d3e service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:46.565161 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f213e0-6cfe-427c-b4f2-2c503efa1d3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:46.573337 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f213e0-6cfe-427c-b4f2-2c503efa1d3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:46.574934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f213e0-6cfe-427c-b4f2-2c503efa1d3e service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:46.578902 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:46.584968 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 680/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:48.857730 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4010397-ee1e-435a-b09f-16a5014b0806 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:48.858600 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4010397-ee1e-435a-b09f-16a5014b0806 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:48.859062 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4010397-ee1e-435a-b09f-16a5014b0806 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:48.860806 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 700/1380] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:37:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:37:48.867006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3d4b4-c618-4aa5-ae9b-0465100b3224 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:48.867830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3d4b4-c618-4aa5-ae9b-0465100b3224 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:48.868349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3d4b4-c618-4aa5-ae9b-0465100b3224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:48.889813 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-7bc3d4b4-c618-4aa5-ae9b-0465100b3224 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:48.891249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-7bc3d4b4-c618-4aa5-ae9b-0465100b3224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:48.901963 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 681/1381] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:37:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:48.951326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4010397-ee1e-435a-b09f-16a5014b0806 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:48.976023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4010397-ee1e-435a-b09f-16a5014b0806 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:48.985340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d855e-19a3-4c29-8283-6ecb1bc1bf6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:48.986295 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d855e-19a3-4c29-8283-6ecb1bc1bf6f service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:48.986796 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d855e-19a3-4c29-8283-6ecb1bc1bf6f service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:48.987311 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d855e-19a3-4c29-8283-6ecb1bc1bf6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:48.998866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d855e-19a3-4c29-8283-6ecb1bc1bf6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:49.000824 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d855e-19a3-4c29-8283-6ecb1bc1bf6f service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:49.011592 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 701/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:51.262148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d091833-edb2-456a-9d49-91c9c3d00b0d None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:51.262639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d091833-edb2-456a-9d49-91c9c3d00b0d None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:51.262759 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d091833-edb2-456a-9d49-91c9c3d00b0d None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:51.264216 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 682/1383] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:51.270373 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f4d4d-a5c4-495e-a97b-485800e08550 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:51.271163 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f4d4d-a5c4-495e-a97b-485800e08550 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:51.271595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f4d4d-a5c4-495e-a97b-485800e08550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:51.291164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-380f4d4d-a5c4-495e-a97b-485800e08550 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:51.292541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-380f4d4d-a5c4-495e-a97b-485800e08550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:51.300462 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 702/1384] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:37:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:51.344631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d091833-edb2-456a-9d49-91c9c3d00b0d None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:51.360843 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d091833-edb2-456a-9d49-91c9c3d00b0d None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:51.366374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c4a5b-a7d8-4577-bdf5-c7937be7e711 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:51.367017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c4a5b-a7d8-4577-bdf5-c7937be7e711 service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:51.367303 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c4a5b-a7d8-4577-bdf5-c7937be7e711 service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:51.367580 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c4a5b-a7d8-4577-bdf5-c7937be7e711 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:51.377291 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c4a5b-a7d8-4577-bdf5-c7937be7e711 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:51.378909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c4a5b-a7d8-4577-bdf5-c7937be7e711 service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:51.389583 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 683/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:53.444780 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd4796-3ed7-4b0f-b3a9-4381d275eead None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:53.445456 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd4796-3ed7-4b0f-b3a9-4381d275eead None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:53.445456 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd4796-3ed7-4b0f-b3a9-4381d275eead None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:53.446598 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 703/1386] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:53.453589 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59bf1e-a974-44de-beb6-df9e2c0d2491 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:53.454053 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59bf1e-a974-44de-beb6-df9e2c0d2491 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:53.454371 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59bf1e-a974-44de-beb6-df9e2c0d2491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:53.477134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-5a59bf1e-a974-44de-beb6-df9e2c0d2491 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:53.478641 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-5a59bf1e-a974-44de-beb6-df9e2c0d2491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:53.488564 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 684/1387] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:53.535567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd4796-3ed7-4b0f-b3a9-4381d275eead None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:53.550387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd4796-3ed7-4b0f-b3a9-4381d275eead None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:53.555858 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934fa2e-0adf-420d-8afa-03a2a4e7fcba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:53.556499 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934fa2e-0adf-420d-8afa-03a2a4e7fcba service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:53.556904 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934fa2e-0adf-420d-8afa-03a2a4e7fcba service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:53.557190 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934fa2e-0adf-420d-8afa-03a2a4e7fcba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:53.566030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934fa2e-0adf-420d-8afa-03a2a4e7fcba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:53.567536 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934fa2e-0adf-420d-8afa-03a2a4e7fcba service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:53.578515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 704/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:55.814235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d0723-9097-4baa-90d4-150e26620d6e None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:55.814786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d0723-9097-4baa-90d4-150e26620d6e None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:55.815206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d0723-9097-4baa-90d4-150e26620d6e None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:55.817521 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 685/1389] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:37:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:55.823544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1df9c-fa6a-459f-a275-40762229807e None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:55.824007 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1df9c-fa6a-459f-a275-40762229807e None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:55.824125 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1df9c-fa6a-459f-a275-40762229807e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:55.844155 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-4cb1df9c-fa6a-459f-a275-40762229807e None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:55.845454 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-4cb1df9c-fa6a-459f-a275-40762229807e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:55.854114 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 705/1390] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:37:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:55.905067 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d0723-9097-4baa-90d4-150e26620d6e None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:55.919925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d0723-9097-4baa-90d4-150e26620d6e None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:55.925526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94f591-a2f8-4e2d-b290-8c95ce107081 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:55.926181 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94f591-a2f8-4e2d-b290-8c95ce107081 service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:55.926474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94f591-a2f8-4e2d-b290-8c95ce107081 service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:55.926759 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94f591-a2f8-4e2d-b290-8c95ce107081 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:55.937708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94f591-a2f8-4e2d-b290-8c95ce107081 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:55.940354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94f591-a2f8-4e2d-b290-8c95ce107081 service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:55.957107 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 686/1391] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:37:58.258114 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a2de2-a112-4ad9-93ff-81a867708150 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:58.258710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a2de2-a112-4ad9-93ff-81a867708150 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:58.258710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a2de2-a112-4ad9-93ff-81a867708150 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:58.260787 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 706/1392] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:37:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:37:58.266714 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a1f6a-eb14-4042-bc7d-c28c71f62dba None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:58.267151 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a1f6a-eb14-4042-bc7d-c28c71f62dba None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:58.267297 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a1f6a-eb14-4042-bc7d-c28c71f62dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:58.285632 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-829a1f6a-eb14-4042-bc7d-c28c71f62dba None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:58.286889 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-829a1f6a-eb14-4042-bc7d-c28c71f62dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:58.295181 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 687/1393] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:58.341717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a2de2-a112-4ad9-93ff-81a867708150 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:58.357483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a2de2-a112-4ad9-93ff-81a867708150 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:58.362903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff020176-2ca8-40ac-8f08-affe71664f1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:58.363554 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff020176-2ca8-40ac-8f08-affe71664f1c service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:58.363839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff020176-2ca8-40ac-8f08-affe71664f1c service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:58.364117 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff020176-2ca8-40ac-8f08-affe71664f1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:58.373096 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff020176-2ca8-40ac-8f08-affe71664f1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:58.374679 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff020176-2ca8-40ac-8f08-affe71664f1c service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:58.383984 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 707/1394] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:00.570627 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2e28d-d98a-48a4-ba58-9b071fd4a58d None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:00.571142 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2e28d-d98a-48a4-ba58-9b071fd4a58d None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:00.571142 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2e28d-d98a-48a4-ba58-9b071fd4a58d None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:00.572877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 688/1395] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:00.578248 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce408bf-9d3c-479e-895d-f71e7444b969 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:00.578713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce408bf-9d3c-479e-895d-f71e7444b969 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:00.578881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce408bf-9d3c-479e-895d-f71e7444b969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:00.596948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-0ce408bf-9d3c-479e-895d-f71e7444b969 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:00.598131 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-0ce408bf-9d3c-479e-895d-f71e7444b969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:00.606404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 708/1396] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:00.650669 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2e28d-d98a-48a4-ba58-9b071fd4a58d None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:00.665198 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2e28d-d98a-48a4-ba58-9b071fd4a58d None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:00.671162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf7eb8-0aec-41ad-9115-18e2c754f047 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:00.671799 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf7eb8-0aec-41ad-9115-18e2c754f047 service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:00.672087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf7eb8-0aec-41ad-9115-18e2c754f047 service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:00.672367 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf7eb8-0aec-41ad-9115-18e2c754f047 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:00.680846 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf7eb8-0aec-41ad-9115-18e2c754f047 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:00.682503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf7eb8-0aec-41ad-9115-18e2c754f047 service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:00.691974 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 689/1397] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:00.722684 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dba4a2-d2d7-4dde-b27f-ecdb94661706 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:00.722978 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dba4a2-d2d7-4dde-b27f-ecdb94661706 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:00.723258 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dba4a2-d2d7-4dde-b27f-ecdb94661706 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:00.724757 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 709/1398] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:00.730795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516627a8-9d50-487b-8a42-1cbcec45bf80 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:00.731093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516627a8-9d50-487b-8a42-1cbcec45bf80 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:00.731377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516627a8-9d50-487b-8a42-1cbcec45bf80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:00.748895 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-516627a8-9d50-487b-8a42-1cbcec45bf80 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:00.750088 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-516627a8-9d50-487b-8a42-1cbcec45bf80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:00.757855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 690/1399] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:00.815372 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116c41c-801e-41d8-8d1b-0eee483c09e9 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:00.815672 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116c41c-801e-41d8-8d1b-0eee483c09e9 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:00.815959 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116c41c-801e-41d8-8d1b-0eee483c09e9 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:00.817362 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 710/1400] 10.222.0.33 () {54 vars in 962 bytes} [Thu Jun 23 22:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:00.823420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5facdb6e-bc17-4ddb-bd45-8bfa57fb37b6 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:00.823671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5facdb6e-bc17-4ddb-bd45-8bfa57fb37b6 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:00.823957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5facdb6e-bc17-4ddb-bd45-8bfa57fb37b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:00.871930 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-5facdb6e-bc17-4ddb-bd45-8bfa57fb37b6 None None] MFA Rules not processed for user `983801bb18924fe4b3150693765eb13b`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:00.873180 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-5facdb6e-bc17-4ddb-bd45-8bfa57fb37b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:00.938431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 691/1401] 10.222.0.33 () {58 vars in 1059 bytes} [Thu Jun 23 22:38:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:00.946357 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116c41c-801e-41d8-8d1b-0eee483c09e9 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:00.958515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116c41c-801e-41d8-8d1b-0eee483c09e9 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:00.962060 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648693bf-37c2-4bb9-b040-e501a87937fa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '983801bb18924fe4b3150693765eb13b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:00.963265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648693bf-37c2-4bb9-b040-e501a87937fa service glance] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:00.963555 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648693bf-37c2-4bb9-b040-e501a87937fa service glance] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:00.963825 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648693bf-37c2-4bb9-b040-e501a87937fa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:00.974312 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648693bf-37c2-4bb9-b040-e501a87937fa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:00.975777 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648693bf-37c2-4bb9-b040-e501a87937fa service glance] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:00.986334 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 711/1402] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:02.119378 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f191d6-47d2-4a2b-8d74-198569f33171 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:02.119901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f191d6-47d2-4a2b-8d74-198569f33171 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:02.120018 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f191d6-47d2-4a2b-8d74-198569f33171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:02.167793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-f8f191d6-47d2-4a2b-8d74-198569f33171 None None] MFA Rules not processed for user `0a7c204a298c42f0bbc32fa9a6ce1f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:02.169063 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-f8f191d6-47d2-4a2b-8d74-198569f33171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:02.209865 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:02.233363 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 692/1403] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 22:38:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:02.274683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95883585-4722-49ca-9b80-e2fa62ce2c7d None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:02.275508 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95883585-4722-49ca-9b80-e2fa62ce2c7d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:02.275508 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95883585-4722-49ca-9b80-e2fa62ce2c7d None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:02.277065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 712/1404] 10.222.0.33 () {54 vars in 942 bytes} [Thu Jun 23 22:38:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:02.283394 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a3852-2b5a-4305-8383-bbab29104f9e None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:02.283694 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a3852-2b5a-4305-8383-bbab29104f9e None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:02.283967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a3852-2b5a-4305-8383-bbab29104f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:02.332596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-906a3852-2b5a-4305-8383-bbab29104f9e None None] MFA Rules not processed for user `b321586cdb3d4d25904472aaa4adba71`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:02.333829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-906a3852-2b5a-4305-8383-bbab29104f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:02.404705 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 693/1405] 10.222.0.33 () {58 vars in 1039 bytes} [Thu Jun 23 22:38:02 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:02.414926 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95883585-4722-49ca-9b80-e2fa62ce2c7d None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:02.422938 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=61 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:02.429242 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95883585-4722-49ca-9b80-e2fa62ce2c7d None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:02.435131 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3006c7-78ab-4a5f-9338-c75c1fbff34e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b321586cdb3d4d25904472aaa4adba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:02.435801 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3006c7-78ab-4a5f-9338-c75c1fbff34e service swift] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:02.436086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3006c7-78ab-4a5f-9338-c75c1fbff34e service swift] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:02.436361 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3006c7-78ab-4a5f-9338-c75c1fbff34e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:02.445085 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3006c7-78ab-4a5f-9338-c75c1fbff34e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:02.454189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3006c7-78ab-4a5f-9338-c75c1fbff34e service swift] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:02.454871 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 713/1406] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 22:38:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:38:02.892557 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6b6a8-8f15-43e7-9025-d9bf00e921c0 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:02.892918 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6b6a8-8f15-43e7-9025-d9bf00e921c0 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:02.893162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6b6a8-8f15-43e7-9025-d9bf00e921c0 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:02.894543 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 694/1407] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:02.900364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533d7b9-129e-40c7-82eb-cece22119e9b None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:02.900811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533d7b9-129e-40c7-82eb-cece22119e9b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:02.901198 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533d7b9-129e-40c7-82eb-cece22119e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:02.919776 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-3533d7b9-129e-40c7-82eb-cece22119e9b None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:02.921116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3533d7b9-129e-40c7-82eb-cece22119e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:02.935177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 714/1408] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:02.973593 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6b6a8-8f15-43e7-9025-d9bf00e921c0 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:02.988637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6b6a8-8f15-43e7-9025-d9bf00e921c0 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:02.994972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001ccdd-1065-46a5-af37-c56620f5a42c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:02.999640 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001ccdd-1065-46a5-af37-c56620f5a42c service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:02.999956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001ccdd-1065-46a5-af37-c56620f5a42c service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:03.000236 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001ccdd-1065-46a5-af37-c56620f5a42c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:03.010420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001ccdd-1065-46a5-af37-c56620f5a42c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:03.012130 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001ccdd-1065-46a5-af37-c56620f5a42c service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:03.022281 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 695/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:12.913967 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=8 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:13.017787 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=59 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:25.829794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6f462-bfb7-4ce3-86c7-22be10ec4a83 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:25.830366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6f462-bfb7-4ce3-86c7-22be10ec4a83 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:25.830366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6f462-bfb7-4ce3-86c7-22be10ec4a83 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:25.831760 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 715/1410] 10.222.0.33 () {54 vars in 918 bytes} [Thu Jun 23 22:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:25.837402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763d664-7f94-4b95-85d9-3a65e3d238cc None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:25.838033 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763d664-7f94-4b95-85d9-3a65e3d238cc None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:25.838033 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763d664-7f94-4b95-85d9-3a65e3d238cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:25.855689 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-c763d664-7f94-4b95-85d9-3a65e3d238cc None None] MFA Rules not processed for user `d3ac44b3f0224e56a15993a2f29c1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:25.856911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c763d664-7f94-4b95-85d9-3a65e3d238cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:25.865289 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 696/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Jun 23 22:38:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:25.880294 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc14d40-f019-488e-a6cf-f01f1d273063 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:25.880643 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc14d40-f019-488e-a6cf-f01f1d273063 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:25.880927 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc14d40-f019-488e-a6cf-f01f1d273063 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:25.882304 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 716/1412] 10.222.0.33 () {54 vars in 918 bytes} [Thu Jun 23 22:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:25.887394 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeacf44-b6df-479b-99b4-aaffdbbf650d None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:25.887690 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeacf44-b6df-479b-99b4-aaffdbbf650d None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:25.887972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeacf44-b6df-479b-99b4-aaffdbbf650d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:25.905993 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-ffeacf44-b6df-479b-99b4-aaffdbbf650d None None] MFA Rules not processed for user `159cc8befc3a4d57bbd313832084db62`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:25.907173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ffeacf44-b6df-479b-99b4-aaffdbbf650d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:25.916337 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 697/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Jun 23 22:38:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:38:28.390804 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29550311-bd4e-4464-8659-9a25a174145b None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:28.391303 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29550311-bd4e-4464-8659-9a25a174145b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:28.391420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29550311-bd4e-4464-8659-9a25a174145b None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:28.393021 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 717/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:28.398190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff7bd7-be40-4029-9156-a97231d6a96a None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:28.399374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff7bd7-be40-4029-9156-a97231d6a96a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:28.399374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff7bd7-be40-4029-9156-a97231d6a96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:28.417640 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-d1ff7bd7-be40-4029-9156-a97231d6a96a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:28.418906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-d1ff7bd7-be40-4029-9156-a97231d6a96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:28.426561 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 698/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:28.469904 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29550311-bd4e-4464-8659-9a25a174145b None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:28.484974 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29550311-bd4e-4464-8659-9a25a174145b None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:28.490490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756152f2-bd22-4940-adc4-36b038ff0e53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:28.491125 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756152f2-bd22-4940-adc4-36b038ff0e53 service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:28.491404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756152f2-bd22-4940-adc4-36b038ff0e53 service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:28.491681 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756152f2-bd22-4940-adc4-36b038ff0e53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:28.500205 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756152f2-bd22-4940-adc4-36b038ff0e53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:28.501815 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756152f2-bd22-4940-adc4-36b038ff0e53 service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:28.511860 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 718/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:29.408740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a662a8-aab4-4836-b563-19732e483267 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:29.409308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a662a8-aab4-4836-b563-19732e483267 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:29.409308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a662a8-aab4-4836-b563-19732e483267 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:29.410659 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 699/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:29.416199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f149afb-abd7-43af-aaa8-58a57aa38871 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:29.416657 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f149afb-abd7-43af-aaa8-58a57aa38871 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:29.416806 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f149afb-abd7-43af-aaa8-58a57aa38871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:29.434828 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-8f149afb-abd7-43af-aaa8-58a57aa38871 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:29.436008 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-8f149afb-abd7-43af-aaa8-58a57aa38871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:29.443649 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 719/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:38:29.592733 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a662a8-aab4-4836-b563-19732e483267 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:29.607080 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a662a8-aab4-4836-b563-19732e483267 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:29.612278 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd5b92-4dce-4e80-9f53-906c4d6695a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:29.612944 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd5b92-4dce-4e80-9f53-906c4d6695a3 service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:29.613225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd5b92-4dce-4e80-9f53-906c4d6695a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:29.613499 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd5b92-4dce-4e80-9f53-906c4d6695a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:29.621970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd5b92-4dce-4e80-9f53-906c4d6695a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:29.623546 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd5b92-4dce-4e80-9f53-906c4d6695a3 service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:29.633031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 700/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:30.514269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27e989-635c-4b79-b995-2220faf48702 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:30.514850 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27e989-635c-4b79-b995-2220faf48702 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:30.514850 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27e989-635c-4b79-b995-2220faf48702 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:30.516465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 720/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:30.521409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1cd09-2915-4229-9e1f-f4560e7b0a0b None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:30.521807 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1cd09-2915-4229-9e1f-f4560e7b0a0b None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:30.521977 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1cd09-2915-4229-9e1f-f4560e7b0a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:30.540760 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-0cd1cd09-2915-4229-9e1f-f4560e7b0a0b None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:30.541896 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-0cd1cd09-2915-4229-9e1f-f4560e7b0a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:30.549583 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 701/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:30.699153 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f27e989-635c-4b79-b995-2220faf48702 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:30.713763 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f27e989-635c-4b79-b995-2220faf48702 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:30.721310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87859d87-37de-41b7-a75a-43f532ff78c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:30.721310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87859d87-37de-41b7-a75a-43f532ff78c6 service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:30.721310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87859d87-37de-41b7-a75a-43f532ff78c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:30.721310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87859d87-37de-41b7-a75a-43f532ff78c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:30.728671 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87859d87-37de-41b7-a75a-43f532ff78c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:30.730232 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87859d87-37de-41b7-a75a-43f532ff78c6 service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:30.739355 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 721/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:31.607109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1bc8c-b8c4-45f8-8b3f-9c48fd67538f None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:31.607637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1bc8c-b8c4-45f8-8b3f-9c48fd67538f None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:31.607637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1bc8c-b8c4-45f8-8b3f-9c48fd67538f None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:31.608999 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 702/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:31.614140 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976d322-33b8-4e0d-8afd-a5cfc41d6fcd None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:31.614587 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976d322-33b8-4e0d-8afd-a5cfc41d6fcd None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:31.614696 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976d322-33b8-4e0d-8afd-a5cfc41d6fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:31.633512 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-2976d322-33b8-4e0d-8afd-a5cfc41d6fcd None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:31.634873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-2976d322-33b8-4e0d-8afd-a5cfc41d6fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:31.642756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 722/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:38:31.795210 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1bc8c-b8c4-45f8-8b3f-9c48fd67538f None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:31.809397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1bc8c-b8c4-45f8-8b3f-9c48fd67538f None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:31.814654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa118c-3d90-49ae-9c76-0261a5ddc54b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:31.815284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa118c-3d90-49ae-9c76-0261a5ddc54b service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:31.815566 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa118c-3d90-49ae-9c76-0261a5ddc54b service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:31.815839 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa118c-3d90-49ae-9c76-0261a5ddc54b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:31.824219 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa118c-3d90-49ae-9c76-0261a5ddc54b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:31.825824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa118c-3d90-49ae-9c76-0261a5ddc54b service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:31.834925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 703/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:32.696419 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cfb04-ff6c-4a04-b7c3-cdc56802595e None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:32.696987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cfb04-ff6c-4a04-b7c3-cdc56802595e None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:32.696987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cfb04-ff6c-4a04-b7c3-cdc56802595e None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:32.698342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 723/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:32.703285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb901d-2375-4df7-b67b-80fec22a6755 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:32.703680 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb901d-2375-4df7-b67b-80fec22a6755 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:32.703847 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb901d-2375-4df7-b67b-80fec22a6755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:32.721955 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-0dbb901d-2375-4df7-b67b-80fec22a6755 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:32.723151 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-0dbb901d-2375-4df7-b67b-80fec22a6755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:32.731621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 704/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:32.882007 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cfb04-ff6c-4a04-b7c3-cdc56802595e None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:32.896669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cfb04-ff6c-4a04-b7c3-cdc56802595e None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:32.902639 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689cac8e-d8a2-4e67-b5eb-5ff8a940c025 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:32.903532 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cac8e-d8a2-4e67-b5eb-5ff8a940c025 service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:32.903818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cac8e-d8a2-4e67-b5eb-5ff8a940c025 service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:32.904091 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cac8e-d8a2-4e67-b5eb-5ff8a940c025 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:32.912228 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689cac8e-d8a2-4e67-b5eb-5ff8a940c025 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:32.914383 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689cac8e-d8a2-4e67-b5eb-5ff8a940c025 service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:32.923530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 724/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:33.794514 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12724b8b-37eb-420c-8a6f-546b9516d434 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:33.795200 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12724b8b-37eb-420c-8a6f-546b9516d434 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:33.795200 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12724b8b-37eb-420c-8a6f-546b9516d434 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:33.796426 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 705/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:33.801285 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a58cd-f1ab-4493-87ab-07152bcdfd4e None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:33.801710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a58cd-f1ab-4493-87ab-07152bcdfd4e None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:33.801852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a58cd-f1ab-4493-87ab-07152bcdfd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:33.821168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-e85a58cd-f1ab-4493-87ab-07152bcdfd4e None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:33.822354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e85a58cd-f1ab-4493-87ab-07152bcdfd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:33.830084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 725/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:33.981296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12724b8b-37eb-420c-8a6f-546b9516d434 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:33.995962 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12724b8b-37eb-420c-8a6f-546b9516d434 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:34.001278 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c5069-3257-4e7d-bb79-a6f333f99a3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:34.001911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c5069-3257-4e7d-bb79-a6f333f99a3d service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:34.002198 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c5069-3257-4e7d-bb79-a6f333f99a3d service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:34.002475 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c5069-3257-4e7d-bb79-a6f333f99a3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:34.010636 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c5069-3257-4e7d-bb79-a6f333f99a3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:34.012211 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c5069-3257-4e7d-bb79-a6f333f99a3d service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:34.021712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 706/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:34.921564 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc3f0b-2725-4f43-aa7f-6616289d9bd8 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:34.922100 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc3f0b-2725-4f43-aa7f-6616289d9bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:34.922483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc3f0b-2725-4f43-aa7f-6616289d9bd8 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:34.923903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 726/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:38:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:34.929377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31eb4b-5d2d-4925-b7f7-486bdb398e5a None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:34.929905 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31eb4b-5d2d-4925-b7f7-486bdb398e5a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:34.930016 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31eb4b-5d2d-4925-b7f7-486bdb398e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:34.949418 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-ae31eb4b-5d2d-4925-b7f7-486bdb398e5a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:34.950665 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ae31eb4b-5d2d-4925-b7f7-486bdb398e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:34.958510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 707/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:35.132181 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc3f0b-2725-4f43-aa7f-6616289d9bd8 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:35.147217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc3f0b-2725-4f43-aa7f-6616289d9bd8 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:35.154244 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0846dc6-5b08-4be5-806b-6137b5279951 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:35.154931 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0846dc6-5b08-4be5-806b-6137b5279951 service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:35.155218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0846dc6-5b08-4be5-806b-6137b5279951 service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:35.155497 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0846dc6-5b08-4be5-806b-6137b5279951 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:35.165491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0846dc6-5b08-4be5-806b-6137b5279951 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:35.167510 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0846dc6-5b08-4be5-806b-6137b5279951 service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:35.179532 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 727/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:36.093105 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a66e5d-4faf-44e7-ae4e-30ab9dc96293 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:36.093682 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a66e5d-4faf-44e7-ae4e-30ab9dc96293 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:36.093682 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a66e5d-4faf-44e7-ae4e-30ab9dc96293 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:36.095078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 708/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:36.099978 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398db745-a5a0-4d96-8ac1-e7fec617656b None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:36.100369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398db745-a5a0-4d96-8ac1-e7fec617656b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:36.100545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398db745-a5a0-4d96-8ac1-e7fec617656b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:36.119076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-398db745-a5a0-4d96-8ac1-e7fec617656b None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:36.120263 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-398db745-a5a0-4d96-8ac1-e7fec617656b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:36.127894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 728/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:36.279448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a66e5d-4faf-44e7-ae4e-30ab9dc96293 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:36.294191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a66e5d-4faf-44e7-ae4e-30ab9dc96293 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:36.299519 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82255e5a-0495-4586-b4d0-7638d9a97bbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:36.300150 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82255e5a-0495-4586-b4d0-7638d9a97bbd service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:36.300434 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82255e5a-0495-4586-b4d0-7638d9a97bbd service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:36.300747 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82255e5a-0495-4586-b4d0-7638d9a97bbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:36.309447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82255e5a-0495-4586-b4d0-7638d9a97bbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:36.311050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82255e5a-0495-4586-b4d0-7638d9a97bbd service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:36.320817 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 709/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:37.174979 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b4e4f-4252-45bb-b925-430ed3ef9391 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:37.175542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b4e4f-4252-45bb-b925-430ed3ef9391 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:37.175542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b4e4f-4252-45bb-b925-430ed3ef9391 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:37.176935 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 729/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:37.182686 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bb3fa-0870-4e61-9511-0bf895d7d121 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:37.183133 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bb3fa-0870-4e61-9511-0bf895d7d121 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:37.183230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bb3fa-0870-4e61-9511-0bf895d7d121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:37.202209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-5b2bb3fa-0870-4e61-9511-0bf895d7d121 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:37.203408 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-5b2bb3fa-0870-4e61-9511-0bf895d7d121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:37.211201 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 710/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:37.361295 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b4e4f-4252-45bb-b925-430ed3ef9391 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:37.376193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b4e4f-4252-45bb-b925-430ed3ef9391 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:37.381596 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef006b-ec96-451b-809e-9cd4464a098b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:37.382342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef006b-ec96-451b-809e-9cd4464a098b service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:37.383172 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef006b-ec96-451b-809e-9cd4464a098b service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:37.383462 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef006b-ec96-451b-809e-9cd4464a098b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:37.392261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef006b-ec96-451b-809e-9cd4464a098b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:37.393880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef006b-ec96-451b-809e-9cd4464a098b service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:37.404177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 730/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:39.353803 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bbb5a-4251-436f-a029-6b1e2b64ff25 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:39.354369 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bbb5a-4251-436f-a029-6b1e2b64ff25 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:39.354369 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bbb5a-4251-436f-a029-6b1e2b64ff25 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:39.355731 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 711/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:39.361126 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef949945-d916-4767-9ac0-3537decd5191 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:39.361540 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef949945-d916-4767-9ac0-3537decd5191 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:39.361681 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef949945-d916-4767-9ac0-3537decd5191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:39.379443 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-ef949945-d916-4767-9ac0-3537decd5191 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:39.380643 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ef949945-d916-4767-9ac0-3537decd5191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:39.388985 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 731/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:39.427007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165bbb5a-4251-436f-a029-6b1e2b64ff25 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:39.438236 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165bbb5a-4251-436f-a029-6b1e2b64ff25 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:39.441834 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed879b5-e3bb-4529-b062-1f31e678dec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:39.442471 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed879b5-e3bb-4529-b062-1f31e678dec3 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:39.442751 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed879b5-e3bb-4529-b062-1f31e678dec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:39.443023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed879b5-e3bb-4529-b062-1f31e678dec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:39.452684 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed879b5-e3bb-4529-b062-1f31e678dec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:39.454116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed879b5-e3bb-4529-b062-1f31e678dec3 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:39.463514 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 712/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:41.651130 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51765e77-dc99-4d87-9845-0900422dca48 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:41.651639 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51765e77-dc99-4d87-9845-0900422dca48 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:41.651639 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51765e77-dc99-4d87-9845-0900422dca48 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:41.653246 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 732/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:41.658640 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507c549-78e2-4c48-9f3f-11859ff96ab8 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:41.659056 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507c549-78e2-4c48-9f3f-11859ff96ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:41.659203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507c549-78e2-4c48-9f3f-11859ff96ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:41.677123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-4507c549-78e2-4c48-9f3f-11859ff96ab8 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:41.678309 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-4507c549-78e2-4c48-9f3f-11859ff96ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:41.686140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 713/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:38:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:41.723406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51765e77-dc99-4d87-9845-0900422dca48 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:41.734811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51765e77-dc99-4d87-9845-0900422dca48 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:41.738312 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4990ad-31bc-450b-ae48-4faa45ea2fbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:41.738946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4990ad-31bc-450b-ae48-4faa45ea2fbf service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:41.739233 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4990ad-31bc-450b-ae48-4faa45ea2fbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:41.739511 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4990ad-31bc-450b-ae48-4faa45ea2fbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:41.748487 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4990ad-31bc-450b-ae48-4faa45ea2fbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:41.749932 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4990ad-31bc-450b-ae48-4faa45ea2fbf service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:41.759243 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 733/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:44.153791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72451baf-0f06-4abc-a969-5f555d412e1c None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:44.154365 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72451baf-0f06-4abc-a969-5f555d412e1c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:44.154365 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72451baf-0f06-4abc-a969-5f555d412e1c None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:44.155735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 714/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:44.160917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57120686-d7cf-4771-97b0-db1035eacbe8 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:44.161368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57120686-d7cf-4771-97b0-db1035eacbe8 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:44.161479 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57120686-d7cf-4771-97b0-db1035eacbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:44.179718 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-57120686-d7cf-4771-97b0-db1035eacbe8 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:44.180958 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-57120686-d7cf-4771-97b0-db1035eacbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:44.188556 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 734/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:38:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:44.227329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72451baf-0f06-4abc-a969-5f555d412e1c None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:44.237919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72451baf-0f06-4abc-a969-5f555d412e1c None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:44.241463 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b863c-3d9a-45d4-882d-337ba2160e57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:44.242096 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b863c-3d9a-45d4-882d-337ba2160e57 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:44.242382 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b863c-3d9a-45d4-882d-337ba2160e57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:44.242661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b863c-3d9a-45d4-882d-337ba2160e57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:44.251395 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b863c-3d9a-45d4-882d-337ba2160e57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:44.252912 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b863c-3d9a-45d4-882d-337ba2160e57 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:44.262693 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 715/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:46.415437 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5b522-a1b5-47dd-98d1-9a97a0f0af1b None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:46.415963 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5b522-a1b5-47dd-98d1-9a97a0f0af1b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:46.416080 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5b522-a1b5-47dd-98d1-9a97a0f0af1b None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:46.417417 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 735/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:46.422493 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dcfc7-aade-4d52-b134-7a7e34066cc7 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:46.422883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dcfc7-aade-4d52-b134-7a7e34066cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:46.423050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dcfc7-aade-4d52-b134-7a7e34066cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:46.440865 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-9f0dcfc7-aade-4d52-b134-7a7e34066cc7 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:46.442043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-9f0dcfc7-aade-4d52-b134-7a7e34066cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:46.450425 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 716/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:46.487669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5b522-a1b5-47dd-98d1-9a97a0f0af1b None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:46.498783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5b522-a1b5-47dd-98d1-9a97a0f0af1b None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:46.502245 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b07ff-5927-4ecd-a723-c926db9243ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:46.502878 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b07ff-5927-4ecd-a723-c926db9243ca service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:46.503160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b07ff-5927-4ecd-a723-c926db9243ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:46.503436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b07ff-5927-4ecd-a723-c926db9243ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:46.512687 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b07ff-5927-4ecd-a723-c926db9243ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:46.514094 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b07ff-5927-4ecd-a723-c926db9243ca service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:46.523523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 736/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:48.621584 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030a962-f36c-43b3-a0cd-ebfba2dc6eaa None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:48.622120 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030a962-f36c-43b3-a0cd-ebfba2dc6eaa None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:48.622120 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030a962-f36c-43b3-a0cd-ebfba2dc6eaa None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:48.623440 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 717/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:48.628859 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2584f1-b0aa-41a0-98b9-59d85d748acd None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:48.629304 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2584f1-b0aa-41a0-98b9-59d85d748acd None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:48.629441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2584f1-b0aa-41a0-98b9-59d85d748acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:48.647206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-0a2584f1-b0aa-41a0-98b9-59d85d748acd None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:48.649166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-0a2584f1-b0aa-41a0-98b9-59d85d748acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:48.656832 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 737/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:48.695546 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030a962-f36c-43b3-a0cd-ebfba2dc6eaa None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:48.706206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030a962-f36c-43b3-a0cd-ebfba2dc6eaa None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:48.709685 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944c1e9-74cb-4930-adab-0f9d448371d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:48.711065 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944c1e9-74cb-4930-adab-0f9d448371d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:48.711351 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944c1e9-74cb-4930-adab-0f9d448371d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:48.711631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944c1e9-74cb-4930-adab-0f9d448371d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:48.720410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944c1e9-74cb-4930-adab-0f9d448371d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:48.721866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944c1e9-74cb-4930-adab-0f9d448371d8 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:48.731997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 718/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:50.808954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fd5d5-7e6e-470a-8188-4ebf35de88a2 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:50.809492 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fd5d5-7e6e-470a-8188-4ebf35de88a2 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:50.809626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fd5d5-7e6e-470a-8188-4ebf35de88a2 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:50.810893 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 738/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:50.816120 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63446e13-e4ce-48ab-888e-1f4ac7fd07cf None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:50.816528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63446e13-e4ce-48ab-888e-1f4ac7fd07cf None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:50.816717 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63446e13-e4ce-48ab-888e-1f4ac7fd07cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:50.834219 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-63446e13-e4ce-48ab-888e-1f4ac7fd07cf None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:50.836035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-63446e13-e4ce-48ab-888e-1f4ac7fd07cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:50.843793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 719/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:50.881439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fd5d5-7e6e-470a-8188-4ebf35de88a2 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:50.892414 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fd5d5-7e6e-470a-8188-4ebf35de88a2 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:50.895884 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecdccdc-c511-4ed7-a6fc-7298f7efd477 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:50.896509 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdccdc-c511-4ed7-a6fc-7298f7efd477 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:50.896822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdccdc-c511-4ed7-a6fc-7298f7efd477 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:50.897100 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdccdc-c511-4ed7-a6fc-7298f7efd477 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:50.906503 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecdccdc-c511-4ed7-a6fc-7298f7efd477 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:50.908022 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecdccdc-c511-4ed7-a6fc-7298f7efd477 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:50.918034 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 739/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:52.994967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b04db-56ea-426f-8dbf-b2c24a5fcecc None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.995526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b04db-56ea-426f-8dbf-b2c24a5fcecc None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.995526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b04db-56ea-426f-8dbf-b2c24a5fcecc None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.996937 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 720/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:53.001984 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ce2e1-ab88-40fe-b383-ac8edc1ec1e3 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:53.002434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ce2e1-ab88-40fe-b383-ac8edc1ec1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:53.002556 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ce2e1-ab88-40fe-b383-ac8edc1ec1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:53.008632 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=70 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:53.020907 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-c40ce2e1-ab88-40fe-b383-ac8edc1ec1e3 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:53.022629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-c40ce2e1-ab88-40fe-b383-ac8edc1ec1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:53.030307 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 740/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:53.067551 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b04db-56ea-426f-8dbf-b2c24a5fcecc None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:53.073019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=73 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:53.078602 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b04db-56ea-426f-8dbf-b2c24a5fcecc None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:53.083403 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8345a6-29e1-40a2-89c2-84720fbe7ff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:53.083403 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8345a6-29e1-40a2-89c2-84720fbe7ff1 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:53.083403 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8345a6-29e1-40a2-89c2-84720fbe7ff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:53.083723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8345a6-29e1-40a2-89c2-84720fbe7ff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:53.093285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8345a6-29e1-40a2-89c2-84720fbe7ff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:53.094703 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8345a6-29e1-40a2-89c2-84720fbe7ff1 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:53.104949 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 721/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:55.173347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be6ff4-de1a-4fa7-bb3f-0dfc60ce9ffa None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:55.173858 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be6ff4-de1a-4fa7-bb3f-0dfc60ce9ffa None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:55.173858 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be6ff4-de1a-4fa7-bb3f-0dfc60ce9ffa None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:55.175179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 741/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:55.180972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fc7db-3536-40bf-8483-4a199df1231f None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:55.181399 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fc7db-3536-40bf-8483-4a199df1231f None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:55.181526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fc7db-3536-40bf-8483-4a199df1231f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:55.199319 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-fc2fc7db-3536-40bf-8483-4a199df1231f None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:55.200505 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-fc2fc7db-3536-40bf-8483-4a199df1231f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:55.208103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 722/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:55.245371 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be6ff4-de1a-4fa7-bb3f-0dfc60ce9ffa None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:55.256149 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be6ff4-de1a-4fa7-bb3f-0dfc60ce9ffa None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:55.260780 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4f624-9c8f-41bd-9ccb-f40354089c7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:55.261424 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4f624-9c8f-41bd-9ccb-f40354089c7c service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:55.261714 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4f624-9c8f-41bd-9ccb-f40354089c7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:55.261998 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4f624-9c8f-41bd-9ccb-f40354089c7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:55.271295 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b4f624-9c8f-41bd-9ccb-f40354089c7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:55.273102 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b4f624-9c8f-41bd-9ccb-f40354089c7c service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:55.282724 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 742/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:57.423822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0d339-b5e1-40c6-b3af-0438b207c7f8 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:57.424384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0d339-b5e1-40c6-b3af-0438b207c7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:57.424384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0d339-b5e1-40c6-b3af-0438b207c7f8 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:57.425815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 723/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:57.431422 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dff37-b8d5-4e50-9c10-81152c06fa9b None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:57.431863 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dff37-b8d5-4e50-9c10-81152c06fa9b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:57.431973 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dff37-b8d5-4e50-9c10-81152c06fa9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:57.449608 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-d01dff37-b8d5-4e50-9c10-81152c06fa9b None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:57.450863 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d01dff37-b8d5-4e50-9c10-81152c06fa9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:57.459076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 743/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:57.496043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0d339-b5e1-40c6-b3af-0438b207c7f8 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:57.506740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0d339-b5e1-40c6-b3af-0438b207c7f8 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:57.510235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3be52f-1b39-4cca-9b2c-a5a85a4118ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:57.510872 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3be52f-1b39-4cca-9b2c-a5a85a4118ef service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:57.511156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3be52f-1b39-4cca-9b2c-a5a85a4118ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:57.511466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3be52f-1b39-4cca-9b2c-a5a85a4118ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:57.520499 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3be52f-1b39-4cca-9b2c-a5a85a4118ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:57.522450 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3be52f-1b39-4cca-9b2c-a5a85a4118ef service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:57.531841 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 724/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:59.564701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202c36e-65cf-4e0b-b613-bb2dcee70196 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:59.565212 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202c36e-65cf-4e0b-b613-bb2dcee70196 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:59.565212 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202c36e-65cf-4e0b-b613-bb2dcee70196 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:59.566666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 744/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:38:59.572488 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fa504-0231-4ddf-8310-e5b23669b370 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:59.572976 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fa504-0231-4ddf-8310-e5b23669b370 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:59.573074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fa504-0231-4ddf-8310-e5b23669b370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:59.591229 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-934fa504-0231-4ddf-8310-e5b23669b370 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:59.592458 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-934fa504-0231-4ddf-8310-e5b23669b370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:59.600400 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 725/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:59.638125 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202c36e-65cf-4e0b-b613-bb2dcee70196 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:59.649263 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202c36e-65cf-4e0b-b613-bb2dcee70196 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:59.652734 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6a541-3a99-45a5-85ca-b36f99370d6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:59.653371 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6a541-3a99-45a5-85ca-b36f99370d6c service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:59.653666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6a541-3a99-45a5-85ca-b36f99370d6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:59.653938 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6a541-3a99-45a5-85ca-b36f99370d6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:59.662764 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6a541-3a99-45a5-85ca-b36f99370d6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:59.664203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6a541-3a99-45a5-85ca-b36f99370d6c service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:59.673462 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 745/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:39:09.586374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=17 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:39:09.669352 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=17 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:39:19.445394 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b18bf-f183-4b86-8e87-6930eab38770 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:19.445967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b18bf-f183-4b86-8e87-6930eab38770 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:19.445967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b18bf-f183-4b86-8e87-6930eab38770 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:19.447357 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 726/1471] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:39:19.452885 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f12b6-8812-4826-aead-5af802b8a086 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:19.453343 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f12b6-8812-4826-aead-5af802b8a086 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:19.453440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f12b6-8812-4826-aead-5af802b8a086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:19.471399 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-704f12b6-8812-4826-aead-5af802b8a086 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:39:19.472637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-704f12b6-8812-4826-aead-5af802b8a086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:19.481515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 746/1472] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:39:19.529539 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52ba20-29e8-4b7e-888f-efb1b951c9c9 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:19.529830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52ba20-29e8-4b7e-888f-efb1b951c9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:19.530114 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52ba20-29e8-4b7e-888f-efb1b951c9c9 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:19.531466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 727/1473] 10.222.0.33 () {54 vars in 962 bytes} [Thu Jun 23 22:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:39:19.536297 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b800392-d148-4bf5-ba83-9233e7f4d3ee None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:19.536692 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b800392-d148-4bf5-ba83-9233e7f4d3ee None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:19.537022 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b800392-d148-4bf5-ba83-9233e7f4d3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:19.561714 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-7b800392-d148-4bf5-ba83-9233e7f4d3ee None None] MFA Rules not processed for user `983801bb18924fe4b3150693765eb13b`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:39:19.561714 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-7b800392-d148-4bf5-ba83-9233e7f4d3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:19.562297 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 747/1474] 10.222.0.33 () {58 vars in 1059 bytes} [Thu Jun 23 22:39:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:39:19.570190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52ba20-29e8-4b7e-888f-efb1b951c9c9 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:19.580712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52ba20-29e8-4b7e-888f-efb1b951c9c9 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:19.585014 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d450501-c859-4f15-b3bd-fe16f3c064df service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '983801bb18924fe4b3150693765eb13b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:19.585640 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d450501-c859-4f15-b3bd-fe16f3c064df service glance] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:19.585923 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d450501-c859-4f15-b3bd-fe16f3c064df service glance] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:19.586199 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d450501-c859-4f15-b3bd-fe16f3c064df service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:19.594835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d450501-c859-4f15-b3bd-fe16f3c064df service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:19.596251 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d450501-c859-4f15-b3bd-fe16f3c064df service glance] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:19.605624 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 728/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:39:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:39:21.698390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8bcdc-b20a-42d4-9f86-1400fa7867ba None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:21.698917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8bcdc-b20a-42d4-9f86-1400fa7867ba None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:21.698917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8bcdc-b20a-42d4-9f86-1400fa7867ba None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:21.700225 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 748/1476] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:39:21.705547 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ec70e-bebe-4149-8f45-f1b67b6e472d None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:21.705999 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ec70e-bebe-4149-8f45-f1b67b6e472d None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:21.706125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ec70e-bebe-4149-8f45-f1b67b6e472d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:21.723578 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-e20ec70e-bebe-4149-8f45-f1b67b6e472d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:39:21.724889 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e20ec70e-bebe-4149-8f45-f1b67b6e472d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:21.732518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 729/1477] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:39:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:39:21.756290 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8bcdc-b20a-42d4-9f86-1400fa7867ba None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:21.768730 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8bcdc-b20a-42d4-9f86-1400fa7867ba None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:21.773155 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c637816-a968-43ce-afbf-9284f16da464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:21.773798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c637816-a968-43ce-afbf-9284f16da464 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:21.774083 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c637816-a968-43ce-afbf-9284f16da464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:21.774360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c637816-a968-43ce-afbf-9284f16da464 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:21.775088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c637816-a968-43ce-afbf-9284f16da464 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:21.776894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: WARNING py.warnings [None req-8c637816-a968-43ce-afbf-9284f16da464 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:39:21.776894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: warnings.warn(msg) Jun 23 22:39:21.776894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: Jun 23 22:39:21.778030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c637816-a968-43ce-afbf-9284f16da464 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:21.797143 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 749/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 23 22:39:21 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:39:23.856674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e84e3b-93c6-4968-b874-cd319bdadd2f None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:23.857221 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e84e3b-93c6-4968-b874-cd319bdadd2f None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:23.857221 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e84e3b-93c6-4968-b874-cd319bdadd2f None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:23.858661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 730/1479] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:39:23.864158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd03ae-44a0-482f-b05e-510aba17cd5c None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:23.864599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd03ae-44a0-482f-b05e-510aba17cd5c None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:23.864896 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd03ae-44a0-482f-b05e-510aba17cd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:23.882714 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-d0dd03ae-44a0-482f-b05e-510aba17cd5c None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:39:23.883935 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d0dd03ae-44a0-482f-b05e-510aba17cd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:23.892051 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 750/1480] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:39:23.936118 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e84e3b-93c6-4968-b874-cd319bdadd2f None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:23.950511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e84e3b-93c6-4968-b874-cd319bdadd2f None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:23.955890 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1924d09-c1e3-4c73-8a8e-de01fa201246 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:23.956532 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1924d09-c1e3-4c73-8a8e-de01fa201246 service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:23.956848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1924d09-c1e3-4c73-8a8e-de01fa201246 service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:23.957144 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1924d09-c1e3-4c73-8a8e-de01fa201246 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:23.965946 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1924d09-c1e3-4c73-8a8e-de01fa201246 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:23.967547 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1924d09-c1e3-4c73-8a8e-de01fa201246 service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:23.977901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 731/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:39:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:39:26.191162 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b64d60-16d2-40ab-bf3f-1cead67c0999 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:26.191756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b64d60-16d2-40ab-bf3f-1cead67c0999 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:26.191756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b64d60-16d2-40ab-bf3f-1cead67c0999 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:26.193809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 751/1482] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:39:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:39:26.199108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82e57e-3606-422a-8c50-f3757c5d75e2 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:26.199551 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82e57e-3606-422a-8c50-f3757c5d75e2 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:26.199675 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82e57e-3606-422a-8c50-f3757c5d75e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:26.217147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-1d82e57e-3606-422a-8c50-f3757c5d75e2 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:39:26.218336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-1d82e57e-3606-422a-8c50-f3757c5d75e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:26.226112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 732/1483] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:39:26.263583 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b64d60-16d2-40ab-bf3f-1cead67c0999 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:26.274551 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b64d60-16d2-40ab-bf3f-1cead67c0999 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:26.279245 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7844561-1dfc-4d28-8ed8-6d8b981e029e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:26.279914 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7844561-1dfc-4d28-8ed8-6d8b981e029e service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:26.280197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7844561-1dfc-4d28-8ed8-6d8b981e029e service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:26.280475 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7844561-1dfc-4d28-8ed8-6d8b981e029e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:26.290664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7844561-1dfc-4d28-8ed8-6d8b981e029e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:26.292118 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7844561-1dfc-4d28-8ed8-6d8b981e029e service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:26.302205 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 752/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:39:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:39:28.442855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80564ee7-89c3-4c60-a21a-d7a28112f016 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:28.443407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80564ee7-89c3-4c60-a21a-d7a28112f016 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:28.443407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80564ee7-89c3-4c60-a21a-d7a28112f016 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:28.444822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 733/1485] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:39:28.449994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb957347-67b8-4269-8a4c-34827579aeeb None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:28.450422 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb957347-67b8-4269-8a4c-34827579aeeb None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:28.450559 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb957347-67b8-4269-8a4c-34827579aeeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:28.468997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-eb957347-67b8-4269-8a4c-34827579aeeb None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:39:28.470200 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-eb957347-67b8-4269-8a4c-34827579aeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:28.478951 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 753/1486] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:39:28.517638 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80564ee7-89c3-4c60-a21a-d7a28112f016 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:28.528981 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80564ee7-89c3-4c60-a21a-d7a28112f016 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:28.532781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91756651-1f5a-441c-8131-2b6d57125383 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:28.533424 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91756651-1f5a-441c-8131-2b6d57125383 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:28.533712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91756651-1f5a-441c-8131-2b6d57125383 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:28.533994 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91756651-1f5a-441c-8131-2b6d57125383 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:28.543456 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91756651-1f5a-441c-8131-2b6d57125383 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:28.544890 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91756651-1f5a-441c-8131-2b6d57125383 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:28.554584 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 734/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:39:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:39:30.680766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2212126-6c44-48e5-a67a-b723d20f4c85 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:30.681295 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2212126-6c44-48e5-a67a-b723d20f4c85 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:30.681295 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2212126-6c44-48e5-a67a-b723d20f4c85 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:30.682685 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 754/1488] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:39:30.687970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae895c-f20e-409b-a278-6e634d27533a None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:30.688408 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae895c-f20e-409b-a278-6e634d27533a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:30.688542 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae895c-f20e-409b-a278-6e634d27533a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:30.706894 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-69ae895c-f20e-409b-a278-6e634d27533a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:39:30.708437 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-69ae895c-f20e-409b-a278-6e634d27533a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:30.716358 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 735/1489] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:39:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:39:30.756984 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2212126-6c44-48e5-a67a-b723d20f4c85 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:30.767892 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2212126-6c44-48e5-a67a-b723d20f4c85 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:30.771662 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7223e22-3ac6-4871-9845-79e9a9455e88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:30.772318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7223e22-3ac6-4871-9845-79e9a9455e88 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:30.772634 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7223e22-3ac6-4871-9845-79e9a9455e88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:30.772917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7223e22-3ac6-4871-9845-79e9a9455e88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:30.782094 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7223e22-3ac6-4871-9845-79e9a9455e88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:30.783504 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7223e22-3ac6-4871-9845-79e9a9455e88 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:30.793841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 755/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:39:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:39:33.190852 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ac6be-9ab5-4098-b59f-8204d6421ee3 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:33.191384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ac6be-9ab5-4098-b59f-8204d6421ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:33.191384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ac6be-9ab5-4098-b59f-8204d6421ee3 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:33.193320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 736/1491] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 23 22:39:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:39:33.198588 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c11e51-6161-4566-bdc2-5889748a4700 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:33.198998 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c11e51-6161-4566-bdc2-5889748a4700 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:33.199159 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c11e51-6161-4566-bdc2-5889748a4700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:33.218634 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-a8c11e51-6161-4566-bdc2-5889748a4700 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:39:33.219841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a8c11e51-6161-4566-bdc2-5889748a4700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:33.227830 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 756/1492] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 23 22:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:39:33.266442 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ac6be-9ab5-4098-b59f-8204d6421ee3 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:33.277583 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ac6be-9ab5-4098-b59f-8204d6421ee3 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:33.281031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ddbaa-a5d6-4dd6-a977-3933cdc507bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:33.281663 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ddbaa-a5d6-4dd6-a977-3933cdc507bc service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:33.281944 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ddbaa-a5d6-4dd6-a977-3933cdc507bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:33.282219 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ddbaa-a5d6-4dd6-a977-3933cdc507bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:33.291317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ddbaa-a5d6-4dd6-a977-3933cdc507bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:33.293273 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ddbaa-a5d6-4dd6-a977-3933cdc507bc service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:33.302846 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 737/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:39:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:39:43.212600 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=33 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:39:43.298430 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=33 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:41:25.826753 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663031b-a673-4d2c-860b-b9718d14676f None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:25.827399 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663031b-a673-4d2c-860b-b9718d14676f None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:25.827510 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663031b-a673-4d2c-860b-b9718d14676f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:25.845291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-3663031b-a673-4d2c-860b-b9718d14676f None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:41:25.846485 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3663031b-a673-4d2c-860b-b9718d14676f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:25.854366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 757/1494] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:41:25.864219 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb464cf-5362-4be8-912c-1cf140371bc8 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:25.864738 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb464cf-5362-4be8-912c-1cf140371bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:25.864854 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb464cf-5362-4be8-912c-1cf140371bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:25.883375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-deb464cf-5362-4be8-912c-1cf140371bc8 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:41:25.884601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-deb464cf-5362-4be8-912c-1cf140371bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:25.892561 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 738/1495] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:41:25.896997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663031b-a673-4d2c-860b-b9718d14676f None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:25.906676 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663031b-a673-4d2c-860b-b9718d14676f None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:25.911190 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335aba7-810e-45e6-9528-8697b1309975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:25.911838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335aba7-810e-45e6-9528-8697b1309975 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:25.912126 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335aba7-810e-45e6-9528-8697b1309975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:25.912406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335aba7-810e-45e6-9528-8697b1309975 admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:25.913170 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c335aba7-810e-45e6-9528-8697b1309975 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:41:25.913482 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c335aba7-810e-45e6-9528-8697b1309975 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:25.914505 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: WARNING py.warnings [None req-c335aba7-810e-45e6-9528-8697b1309975 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:41:25.914505 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: warnings.warn(msg) Jun 23 22:41:25.914505 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: Jun 23 22:41:25.915410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c335aba7-810e-45e6-9528-8697b1309975 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:25.927580 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 758/1496] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:41:25.936088 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb464cf-5362-4be8-912c-1cf140371bc8 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:25.945417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb464cf-5362-4be8-912c-1cf140371bc8 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:25.949961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff663ac-baf7-4ff0-8bf2-8823e31c00cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:25.950602 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff663ac-baf7-4ff0-8bf2-8823e31c00cc admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:25.950928 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff663ac-baf7-4ff0-8bf2-8823e31c00cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:25.951389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff663ac-baf7-4ff0-8bf2-8823e31c00cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:25.952072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff663ac-baf7-4ff0-8bf2-8823e31c00cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:25.953923 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: WARNING py.warnings [None req-4ff663ac-baf7-4ff0-8bf2-8823e31c00cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:41:25.953923 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: warnings.warn(msg) Jun 23 22:41:25.953923 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: Jun 23 22:41:25.955118 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff663ac-baf7-4ff0-8bf2-8823e31c00cc admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:26.058093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 739/1497] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:41:25 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:41:26.062318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335aba7-810e-45e6-9528-8697b1309975 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:26.072180 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335aba7-810e-45e6-9528-8697b1309975 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:26.126948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee958d-5ab2-4a4f-93ad-2e61e0a68489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:26.127541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee958d-5ab2-4a4f-93ad-2e61e0a68489 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:26.127824 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee958d-5ab2-4a4f-93ad-2e61e0a68489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:26.128099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee958d-5ab2-4a4f-93ad-2e61e0a68489 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:26.128812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee958d-5ab2-4a4f-93ad-2e61e0a68489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:26.129812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: WARNING py.warnings [None req-4bee958d-5ab2-4a4f-93ad-2e61e0a68489 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:41:26.129812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: warnings.warn(msg) Jun 23 22:41:26.129812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: Jun 23 22:41:26.131020 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee958d-5ab2-4a4f-93ad-2e61e0a68489 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:26.241124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 759/1498] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:41:26 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:41:26.245173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff663ac-baf7-4ff0-8bf2-8823e31c00cc admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:26.256607 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff663ac-baf7-4ff0-8bf2-8823e31c00cc admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:26.261529 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b118d-2d45-451c-8d62-f86b1314fd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:26.262224 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b118d-2d45-451c-8d62-f86b1314fd40 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:26.262569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b118d-2d45-451c-8d62-f86b1314fd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:26.262845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b118d-2d45-451c-8d62-f86b1314fd40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:26.263597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b118d-2d45-451c-8d62-f86b1314fd40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:26.264857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: WARNING py.warnings [None req-819b118d-2d45-451c-8d62-f86b1314fd40 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:41:26.264857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: warnings.warn(msg) Jun 23 22:41:26.264857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: Jun 23 22:41:26.265788 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b118d-2d45-451c-8d62-f86b1314fd40 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:26.281713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 740/1499] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:41:26.286022 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee958d-5ab2-4a4f-93ad-2e61e0a68489 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:26.297095 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee958d-5ab2-4a4f-93ad-2e61e0a68489 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:26.301762 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca84b0-677d-47c0-9554-41b99d53f617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:26.302458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca84b0-677d-47c0-9554-41b99d53f617 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:26.302943 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca84b0-677d-47c0-9554-41b99d53f617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:26.303226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca84b0-677d-47c0-9554-41b99d53f617 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:26.303972 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca84b0-677d-47c0-9554-41b99d53f617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:26.305034 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: WARNING py.warnings [None req-07ca84b0-677d-47c0-9554-41b99d53f617 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:41:26.305034 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: warnings.warn(msg) Jun 23 22:41:26.305034 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: Jun 23 22:41:26.305928 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca84b0-677d-47c0-9554-41b99d53f617 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:26.320756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 760/1500] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:41:26.325125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b118d-2d45-451c-8d62-f86b1314fd40 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:26.335435 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b118d-2d45-451c-8d62-f86b1314fd40 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:26.339954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d6c71-5510-42b2-93da-3f81d5a7b811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:26.340894 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d6c71-5510-42b2-93da-3f81d5a7b811 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:26.341178 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d6c71-5510-42b2-93da-3f81d5a7b811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:26.341973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d6c71-5510-42b2-93da-3f81d5a7b811 admin admin] PATH_INFO: `/v3/projects/dc242883c157431a8083bfaa439a0ca9/users/1d9ef9d78084496fa87633c74f16db63/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:26.358777 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d6c71-5510-42b2-93da-3f81d5a7b811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc242883c157431a8083bfaa439a0ca9, user_id=1d9ef9d78084496fa87633c74f16db63, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:26.359815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: WARNING py.warnings [None req-352d6c71-5510-42b2-93da-3f81d5a7b811 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:41:26.359815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: warnings.warn(msg) Jun 23 22:41:26.359815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: Jun 23 22:41:26.361832 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d6c71-5510-42b2-93da-3f81d5a7b811 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:26.452692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 741/1501] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:41:26 2022] PUT /identity/v3/projects/dc242883c157431a8083bfaa439a0ca9/users/1d9ef9d78084496fa87633c74f16db63/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:41:26.463765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffa90c-1db9-4633-9562-431a786ae811 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:26.464057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffa90c-1db9-4633-9562-431a786ae811 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:26.464337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffa90c-1db9-4633-9562-431a786ae811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:26.490859 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-e9ffa90c-1db9-4633-9562-431a786ae811 None None] MFA Rules not processed for user `1d9ef9d78084496fa87633c74f16db63`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:41:26.492055 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e9ffa90c-1db9-4633-9562-431a786ae811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:26.560449 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 761/1502] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:41:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:41:26.573870 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93e889-c6f2-46da-8db0-77f0c8553f3a None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:26.588521 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93e889-c6f2-46da-8db0-77f0c8553f3a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:26.588521 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93e889-c6f2-46da-8db0-77f0c8553f3a None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:26.588521 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 742/1503] 10.222.0.33 () {54 vars in 963 bytes} [Thu Jun 23 22:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:41:26.588858 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb701ef0-7ae6-4348-aff9-a57b9be72c69 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:26.588858 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb701ef0-7ae6-4348-aff9-a57b9be72c69 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:26.588858 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb701ef0-7ae6-4348-aff9-a57b9be72c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:26.599513 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-fb701ef0-7ae6-4348-aff9-a57b9be72c69 None None] MFA Rules not processed for user `5d1911bfac984e058b0b11bb6400e782`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:41:26.600728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-fb701ef0-7ae6-4348-aff9-a57b9be72c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:26.643830 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:41:26.651098 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 762/1504] 10.222.0.33 () {58 vars in 1060 bytes} [Thu Jun 23 22:41:26 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:41:26.659750 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93e889-c6f2-46da-8db0-77f0c8553f3a None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:26.670130 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93e889-c6f2-46da-8db0-77f0c8553f3a None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:26.673623 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227213e-a57e-4d35-8a5f-6d0ab14ac1c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:26.674294 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227213e-a57e-4d35-8a5f-6d0ab14ac1c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:26.674580 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227213e-a57e-4d35-8a5f-6d0ab14ac1c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:26.674854 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227213e-a57e-4d35-8a5f-6d0ab14ac1c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:26.683344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227213e-a57e-4d35-8a5f-6d0ab14ac1c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:26.684803 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227213e-a57e-4d35-8a5f-6d0ab14ac1c2 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:26.693514 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 743/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:41:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:41:26.740376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4692a-1c8b-458b-a62d-9ffcad31e3ee None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:26.740691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4692a-1c8b-458b-a62d-9ffcad31e3ee None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:26.740976 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4692a-1c8b-458b-a62d-9ffcad31e3ee None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:26.742243 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 763/1506] 10.222.0.33 () {52 vars in 829 bytes} [Thu Jun 23 22:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:41:26.745750 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227213e-a57e-4d35-8a5f-6d0ab14ac1c2 service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:26.809156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227213e-a57e-4d35-8a5f-6d0ab14ac1c2 service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:26.813976 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e574425-f5a3-4732-9cdb-b21e85d0bba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:26.814849 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e574425-f5a3-4732-9cdb-b21e85d0bba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:26.815130 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e574425-f5a3-4732-9cdb-b21e85d0bba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:26.815407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e574425-f5a3-4732-9cdb-b21e85d0bba7 admin admin] PATH_INFO: `/v3/users/1d9ef9d78084496fa87633c74f16db63` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:26.816645 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e574425-f5a3-4732-9cdb-b21e85d0bba7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d9ef9d78084496fa87633c74f16db63)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:26.817673 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: WARNING py.warnings [None req-6e574425-f5a3-4732-9cdb-b21e85d0bba7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:41:26.817673 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: warnings.warn(msg) Jun 23 22:41:26.817673 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: Jun 23 22:41:26.818915 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e574425-f5a3-4732-9cdb-b21e85d0bba7 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:27.118746 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6e574425-f5a3-4732-9cdb-b21e85d0bba7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d9ef9d78084496fa87633c74f16db63'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:41:27.137885 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6e574425-f5a3-4732-9cdb-b21e85d0bba7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d9ef9d78084496fa87633c74f16db63'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:41:27.201893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6e574425-f5a3-4732-9cdb-b21e85d0bba7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d9ef9d78084496fa87633c74f16db63'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:41:27.202524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6e574425-f5a3-4732-9cdb-b21e85d0bba7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d9ef9d78084496fa87633c74f16db63'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:41:27.203229 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6e574425-f5a3-4732-9cdb-b21e85d0bba7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d9ef9d78084496fa87633c74f16db63'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:41:27.212378 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:41:27.222953 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 744/1507] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:41:26 2022] DELETE /identity/v3/users/1d9ef9d78084496fa87633c74f16db63 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:41:27.227459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed19297-f4a7-4c45-a62e-812fae767d63 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:27.227954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed19297-f4a7-4c45-a62e-812fae767d63 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:27.228065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed19297-f4a7-4c45-a62e-812fae767d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:27.245621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-7ed19297-f4a7-4c45-a62e-812fae767d63 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:41:27.246754 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-7ed19297-f4a7-4c45-a62e-812fae767d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:27.307278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 764/1508] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:41:27.318743 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e574425-f5a3-4732-9cdb-b21e85d0bba7 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:27.320163 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-6e574425-f5a3-4732-9cdb-b21e85d0bba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:27.377114 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e574425-f5a3-4732-9cdb-b21e85d0bba7 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:27.380721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47f4fe-2306-4c50-a22f-7ed707170d3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:27.381404 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47f4fe-2306-4c50-a22f-7ed707170d3e service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:27.381826 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47f4fe-2306-4c50-a22f-7ed707170d3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:27.382108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47f4fe-2306-4c50-a22f-7ed707170d3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:27.391374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47f4fe-2306-4c50-a22f-7ed707170d3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:27.392874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47f4fe-2306-4c50-a22f-7ed707170d3e service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:27.402094 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 745/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:41:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:41:27.916769 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed19297-f4a7-4c45-a62e-812fae767d63 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:27.921206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-7ed19297-f4a7-4c45-a62e-812fae767d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:27.931424 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed19297-f4a7-4c45-a62e-812fae767d63 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:27.936420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da3296-ff2f-4cc9-a7b6-27e016fce809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:27.937332 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da3296-ff2f-4cc9-a7b6-27e016fce809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:27.937664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da3296-ff2f-4cc9-a7b6-27e016fce809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:27.937886 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da3296-ff2f-4cc9-a7b6-27e016fce809 admin admin] PATH_INFO: `/v3/projects/dc242883c157431a8083bfaa439a0ca9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:27.939328 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da3296-ff2f-4cc9-a7b6-27e016fce809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc242883c157431a8083bfaa439a0ca9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:27.940208 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: WARNING py.warnings [None req-77da3296-ff2f-4cc9-a7b6-27e016fce809 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:41:27.940208 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: warnings.warn(msg) Jun 23 22:41:27.940208 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: Jun 23 22:41:27.941440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da3296-ff2f-4cc9-a7b6-27e016fce809 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:28.040405 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-77da3296-ff2f-4cc9-a7b6-27e016fce809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc242883c157431a8083bfaa439a0ca9', 'request_id': 'req-77da3296-ff2f-4cc9-a7b6-27e016fce809'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:41:28.055460 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-77da3296-ff2f-4cc9-a7b6-27e016fce809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc242883c157431a8083bfaa439a0ca9', 'request_id': 'req-77da3296-ff2f-4cc9-a7b6-27e016fce809'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:41:28.061520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-77da3296-ff2f-4cc9-a7b6-27e016fce809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc242883c157431a8083bfaa439a0ca9', 'request_id': 'req-77da3296-ff2f-4cc9-a7b6-27e016fce809'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:41:28.109182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-77da3296-ff2f-4cc9-a7b6-27e016fce809 admin admin] The token cache is being invalidate because project dc242883c157431a8083bfaa439a0ca9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:41:28.109473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-77da3296-ff2f-4cc9-a7b6-27e016fce809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:41:28.110173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-77da3296-ff2f-4cc9-a7b6-27e016fce809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:41:28.111891 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 765/1510] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:41:27 2022] DELETE /identity/v3/projects/dc242883c157431a8083bfaa439a0ca9 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:41:37.398122 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=29 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:41:38.105422 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:08.446496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd61d5-38d0-4ec8-85d9-f9e34a998a9f None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:08.447075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd61d5-38d0-4ec8-85d9-f9e34a998a9f None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:08.447075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd61d5-38d0-4ec8-85d9-f9e34a998a9f None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:08.448422 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 746/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:43:08.453802 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d63754-1f57-4c82-9426-094878085917 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:08.454217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d63754-1f57-4c82-9426-094878085917 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:08.454360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d63754-1f57-4c82-9426-094878085917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:08.472072 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-e0d63754-1f57-4c82-9426-094878085917 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:08.473301 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e0d63754-1f57-4c82-9426-094878085917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:08.531552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 766/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:43:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:43:08.688459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd61d5-38d0-4ec8-85d9-f9e34a998a9f None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:08.693988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-d2fd61d5-38d0-4ec8-85d9-f9e34a998a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:08.764645 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd61d5-38d0-4ec8-85d9-f9e34a998a9f None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:08.770014 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188bd3c-10c7-428b-8e67-b1f5a682a44e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:08.770657 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188bd3c-10c7-428b-8e67-b1f5a682a44e service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:08.770949 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188bd3c-10c7-428b-8e67-b1f5a682a44e service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:08.771234 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188bd3c-10c7-428b-8e67-b1f5a682a44e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:08.779273 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188bd3c-10c7-428b-8e67-b1f5a682a44e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:08.780933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188bd3c-10c7-428b-8e67-b1f5a682a44e service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:08.790277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 747/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:09.622487 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b12f96-b28b-4d54-bb91-69c5576003c6 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:09.623019 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b12f96-b28b-4d54-bb91-69c5576003c6 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:09.623019 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b12f96-b28b-4d54-bb91-69c5576003c6 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:09.624592 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 767/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:43:09.629625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a6bc4-84c1-4899-b8c8-6c44ef7c2644 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:09.630153 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a6bc4-84c1-4899-b8c8-6c44ef7c2644 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:09.630300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a6bc4-84c1-4899-b8c8-6c44ef7c2644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:09.648170 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-587a6bc4-84c1-4899-b8c8-6c44ef7c2644 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:09.649394 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-587a6bc4-84c1-4899-b8c8-6c44ef7c2644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:09.657651 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 748/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:09.811147 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b12f96-b28b-4d54-bb91-69c5576003c6 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:09.825349 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b12f96-b28b-4d54-bb91-69c5576003c6 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:09.831529 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e760e-c5cb-42af-aa68-3447824fdbf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:09.832170 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e760e-c5cb-42af-aa68-3447824fdbf0 service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:09.832455 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e760e-c5cb-42af-aa68-3447824fdbf0 service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:09.832762 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e760e-c5cb-42af-aa68-3447824fdbf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:09.840752 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e760e-c5cb-42af-aa68-3447824fdbf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:09.842681 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e760e-c5cb-42af-aa68-3447824fdbf0 service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:09.851884 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 768/1516] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:11.732936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a71d62-1ccd-4670-841f-06083a08e3fd None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:11.733507 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a71d62-1ccd-4670-841f-06083a08e3fd None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:11.733507 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a71d62-1ccd-4670-841f-06083a08e3fd None None] PATH_INFO: `/v3` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:11.734805 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 749/1517] 10.222.0.13 () {54 vars in 925 bytes} [Thu Jun 23 22:43:11 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:43:11.741579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3883b7-ba2b-4777-9489-10dddf3a096a None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:11.741980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3883b7-ba2b-4777-9489-10dddf3a096a None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:11.742149 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3883b7-ba2b-4777-9489-10dddf3a096a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:11.761122 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-cc3883b7-ba2b-4777-9489-10dddf3a096a None None] MFA Rules not processed for user `d3ac44b3f0224e56a15993a2f29c1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:11.762336 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-cc3883b7-ba2b-4777-9489-10dddf3a096a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:11.825183 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 769/1518] 10.222.0.13 () {58 vars in 1015 bytes} [Thu Jun 23 22:43:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:16.117641 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee9a5b-33aa-4d80-abdb-c62070f4343c None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:16.118185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee9a5b-33aa-4d80-abdb-c62070f4343c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:16.118185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee9a5b-33aa-4d80-abdb-c62070f4343c None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:16.119494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 750/1519] 10.222.0.33 () {54 vars in 960 bytes} [Thu Jun 23 22:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:43:16.124533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54650082-677f-4dd4-97c5-c61cabd1890b None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:16.125018 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54650082-677f-4dd4-97c5-c61cabd1890b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:16.125157 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54650082-677f-4dd4-97c5-c61cabd1890b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:16.143623 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-54650082-677f-4dd4-97c5-c61cabd1890b None None] MFA Rules not processed for user `d3ac44b3f0224e56a15993a2f29c1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:16.144843 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-54650082-677f-4dd4-97c5-c61cabd1890b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:16.153193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 770/1520] 10.222.0.33 () {58 vars in 1057 bytes} [Thu Jun 23 22:43:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:16.161309 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee9a5b-33aa-4d80-abdb-c62070f4343c None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:16.174514 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee9a5b-33aa-4d80-abdb-c62070f4343c None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:16.179413 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb37154-01b6-4db0-9d17-ee87d1b4a0a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ac44b3f0224e56a15993a2f29c1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:16.180043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb37154-01b6-4db0-9d17-ee87d1b4a0a7 service placement] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:16.180331 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb37154-01b6-4db0-9d17-ee87d1b4a0a7 service placement] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:16.180667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb37154-01b6-4db0-9d17-ee87d1b4a0a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:16.189070 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb37154-01b6-4db0-9d17-ee87d1b4a0a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:16.190671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb37154-01b6-4db0-9d17-ee87d1b4a0a7 service placement] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:16.200139 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 751/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:21.079449 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4266f9-a85d-4687-b3db-e357c4f7dcb0 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:21.080011 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4266f9-a85d-4687-b3db-e357c4f7dcb0 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:21.080125 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4266f9-a85d-4687-b3db-e357c4f7dcb0 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:21.081513 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 771/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:43:21.086543 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138de3a-7d5b-4c66-b5b1-b7f33ab41b0e None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:21.086968 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138de3a-7d5b-4c66-b5b1-b7f33ab41b0e None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:21.087139 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138de3a-7d5b-4c66-b5b1-b7f33ab41b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:21.105954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-8138de3a-7d5b-4c66-b5b1-b7f33ab41b0e None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:21.107137 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-8138de3a-7d5b-4c66-b5b1-b7f33ab41b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:21.115903 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 752/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:43:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:21.274109 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4266f9-a85d-4687-b3db-e357c4f7dcb0 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:21.289564 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4266f9-a85d-4687-b3db-e357c4f7dcb0 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:21.294564 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d0a35-6d4e-4c6c-89b8-e24ab6abd9f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:21.295199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d0a35-6d4e-4c6c-89b8-e24ab6abd9f5 service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:21.295652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d0a35-6d4e-4c6c-89b8-e24ab6abd9f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:21.296156 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d0a35-6d4e-4c6c-89b8-e24ab6abd9f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:21.304717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d0a35-6d4e-4c6c-89b8-e24ab6abd9f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:21.306337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d0a35-6d4e-4c6c-89b8-e24ab6abd9f5 service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:21.315947 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 772/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:22.134724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57e917-007d-49e7-b2ff-9728dc940558 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:22.135260 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57e917-007d-49e7-b2ff-9728dc940558 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:22.135379 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57e917-007d-49e7-b2ff-9728dc940558 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:22.136827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 753/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:43:22.142083 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433f7a2-eccd-406e-8f69-0af5218e264c None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:22.142539 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433f7a2-eccd-406e-8f69-0af5218e264c None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:22.142679 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433f7a2-eccd-406e-8f69-0af5218e264c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:22.162509 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-5433f7a2-eccd-406e-8f69-0af5218e264c None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:22.163732 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-5433f7a2-eccd-406e-8f69-0af5218e264c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:22.172701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 773/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:22.331742 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57e917-007d-49e7-b2ff-9728dc940558 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:22.346720 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57e917-007d-49e7-b2ff-9728dc940558 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:22.352111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbac62d-d36e-4ba2-a695-2b029b4bc294 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:22.352793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbac62d-d36e-4ba2-a695-2b029b4bc294 service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:22.353112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbac62d-d36e-4ba2-a695-2b029b4bc294 service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:22.353396 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbac62d-d36e-4ba2-a695-2b029b4bc294 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:22.362933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbac62d-d36e-4ba2-a695-2b029b4bc294 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:22.364544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbac62d-d36e-4ba2-a695-2b029b4bc294 service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:22.374360 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 754/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:26.270661 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d6b52-1bfd-431e-9b84-cb0571c87cb3 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:26.271229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d6b52-1bfd-431e-9b84-cb0571c87cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:26.271470 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d6b52-1bfd-431e-9b84-cb0571c87cb3 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:26.272980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 774/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:43:26.278648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bd676-1f97-4513-a21e-89066610c928 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:26.279079 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bd676-1f97-4513-a21e-89066610c928 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:26.279211 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bd676-1f97-4513-a21e-89066610c928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:26.298925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-004bd676-1f97-4513-a21e-89066610c928 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:26.300193 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-004bd676-1f97-4513-a21e-89066610c928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:26.307978 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 755/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:26.346317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d6b52-1bfd-431e-9b84-cb0571c87cb3 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:26.347996 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-847d6b52-1bfd-431e-9b84-cb0571c87cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:26.406657 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d6b52-1bfd-431e-9b84-cb0571c87cb3 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:26.410462 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85004786-1a4d-4b84-8685-2e9860dae903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:26.411107 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85004786-1a4d-4b84-8685-2e9860dae903 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:26.411394 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85004786-1a4d-4b84-8685-2e9860dae903 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:26.411673 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85004786-1a4d-4b84-8685-2e9860dae903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:26.420454 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85004786-1a4d-4b84-8685-2e9860dae903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:26.422706 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85004786-1a4d-4b84-8685-2e9860dae903 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:26.427856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:26.433047 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 775/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:28.452355 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae7205-42bc-4607-940c-cb9e41884fc4 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:28.453503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae7205-42bc-4607-940c-cb9e41884fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:28.453503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae7205-42bc-4607-940c-cb9e41884fc4 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:28.454830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 756/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:43:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:43:28.459875 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88059d8c-1426-4b95-a641-5745b4d6e5e4 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:28.460301 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88059d8c-1426-4b95-a641-5745b4d6e5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:28.460441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88059d8c-1426-4b95-a641-5745b4d6e5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:28.478478 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-88059d8c-1426-4b95-a641-5745b4d6e5e4 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:28.479722 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-88059d8c-1426-4b95-a641-5745b4d6e5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:28.487919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 776/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:28.524197 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae7205-42bc-4607-940c-cb9e41884fc4 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:28.535351 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae7205-42bc-4607-940c-cb9e41884fc4 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:28.538817 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2087596-1456-459b-80a1-ecdf3c22fdcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:28.539387 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2087596-1456-459b-80a1-ecdf3c22fdcb service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:28.539669 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2087596-1456-459b-80a1-ecdf3c22fdcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:28.539945 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2087596-1456-459b-80a1-ecdf3c22fdcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:28.548690 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2087596-1456-459b-80a1-ecdf3c22fdcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:28.550108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2087596-1456-459b-80a1-ecdf3c22fdcb service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:28.559237 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 757/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:32.082699 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8761a5-9115-4aa9-bbda-8526c4cfb3de None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:32.091555 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8761a5-9115-4aa9-bbda-8526c4cfb3de None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:32.091555 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8761a5-9115-4aa9-bbda-8526c4cfb3de None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:32.091555 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 777/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:43:32.091839 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b63a81-1f3d-4cf4-96c1-20871089ca16 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:32.091839 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b63a81-1f3d-4cf4-96c1-20871089ca16 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:32.091839 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b63a81-1f3d-4cf4-96c1-20871089ca16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:32.108553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-41b63a81-1f3d-4cf4-96c1-20871089ca16 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:32.109807 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-41b63a81-1f3d-4cf4-96c1-20871089ca16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:32.117738 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 758/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:32.422925 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8761a5-9115-4aa9-bbda-8526c4cfb3de None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:32.424629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-bd8761a5-9115-4aa9-bbda-8526c4cfb3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:32.480987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8761a5-9115-4aa9-bbda-8526c4cfb3de None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:32.485065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746e71f-3c33-4ee3-8e83-a5ec6a75d38c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:32.485717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746e71f-3c33-4ee3-8e83-a5ec6a75d38c service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:32.486003 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746e71f-3c33-4ee3-8e83-a5ec6a75d38c service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:32.486277 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746e71f-3c33-4ee3-8e83-a5ec6a75d38c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:32.495544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746e71f-3c33-4ee3-8e83-a5ec6a75d38c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:32.497047 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746e71f-3c33-4ee3-8e83-a5ec6a75d38c service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:32.507616 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 778/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:34.559381 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f28c1d-b324-41af-b6ce-a6159c8419f1 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:34.559906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f28c1d-b324-41af-b6ce-a6159c8419f1 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:34.559906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f28c1d-b324-41af-b6ce-a6159c8419f1 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:34.561405 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 759/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:43:34.566690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c61859-2928-4d99-8803-054f1e5d934d None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:34.567177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c61859-2928-4d99-8803-054f1e5d934d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:34.567274 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c61859-2928-4d99-8803-054f1e5d934d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:34.585031 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-26c61859-2928-4d99-8803-054f1e5d934d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:34.586234 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-26c61859-2928-4d99-8803-054f1e5d934d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:34.594589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 779/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:34.895677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f28c1d-b324-41af-b6ce-a6159c8419f1 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:34.897460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-d7f28c1d-b324-41af-b6ce-a6159c8419f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:34.911051 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f28c1d-b324-41af-b6ce-a6159c8419f1 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:34.914516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24749a85-c45c-48f7-8f37-9e526be82c3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:34.915150 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24749a85-c45c-48f7-8f37-9e526be82c3d service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:34.915432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24749a85-c45c-48f7-8f37-9e526be82c3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:34.915711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24749a85-c45c-48f7-8f37-9e526be82c3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:34.924987 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24749a85-c45c-48f7-8f37-9e526be82c3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:34.926394 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24749a85-c45c-48f7-8f37-9e526be82c3d service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:34.935494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 760/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:44.579867 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=32 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:44.931555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=70 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:02.353007 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8153467-7636-44bf-b8eb-06008e0700ec None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:02.355664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8153467-7636-44bf-b8eb-06008e0700ec None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:02.355664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8153467-7636-44bf-b8eb-06008e0700ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:02.379422 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-e8153467-7636-44bf-b8eb-06008e0700ec None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:02.380695 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e8153467-7636-44bf-b8eb-06008e0700ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:02.396243 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 780/1540] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:02.411111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06be5e-6583-4a61-8ec9-e3d9093f05f3 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:02.411707 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06be5e-6583-4a61-8ec9-e3d9093f05f3 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:02.411707 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06be5e-6583-4a61-8ec9-e3d9093f05f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:02.444762 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-3c06be5e-6583-4a61-8ec9-e3d9093f05f3 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:02.446019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3c06be5e-6583-4a61-8ec9-e3d9093f05f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:02.464863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 761/1541] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:02.476637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8153467-7636-44bf-b8eb-06008e0700ec None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:02.488031 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8153467-7636-44bf-b8eb-06008e0700ec None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:02.497397 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8184b88-b738-4e29-bfb4-5ff3e7bfee2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:02.498101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8184b88-b738-4e29-bfb4-5ff3e7bfee2f admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:02.498389 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8184b88-b738-4e29-bfb4-5ff3e7bfee2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:02.498669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8184b88-b738-4e29-bfb4-5ff3e7bfee2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:02.499666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8184b88-b738-4e29-bfb4-5ff3e7bfee2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:45:02.499974 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8184b88-b738-4e29-bfb4-5ff3e7bfee2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:02.503135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23862df8-a2f5-47c9-aaa9-4cb4ba5521e2 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:02.503424 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23862df8-a2f5-47c9-aaa9-4cb4ba5521e2 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:02.503704 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23862df8-a2f5-47c9-aaa9-4cb4ba5521e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:02.515425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8184b88-b738-4e29-bfb4-5ff3e7bfee2f admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:02.551069 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 781/1542] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:45:02.562583 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8184b88-b738-4e29-bfb4-5ff3e7bfee2f admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:02.582588 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8184b88-b738-4e29-bfb4-5ff3e7bfee2f admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:02.591740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9595975-0e05-4211-a24a-1905f4a46458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:02.592433 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9595975-0e05-4211-a24a-1905f4a46458 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:02.592753 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9595975-0e05-4211-a24a-1905f4a46458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:02.593033 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9595975-0e05-4211-a24a-1905f4a46458 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:02.593755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9595975-0e05-4211-a24a-1905f4a46458 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:02.594810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: WARNING py.warnings [None req-f9595975-0e05-4211-a24a-1905f4a46458 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:45:02.594810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: warnings.warn(msg) Jun 23 22:45:02.594810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: Jun 23 22:45:02.596076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9595975-0e05-4211-a24a-1905f4a46458 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:02.725002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-23862df8-a2f5-47c9-aaa9-4cb4ba5521e2 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:02.726233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-23862df8-a2f5-47c9-aaa9-4cb4ba5521e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:02.742303 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 782/1543] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:02 2022] POST /identity/v3/projects => generated 367 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:45:02.744018 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2eec5-a23a-4612-9053-c74c312c3f49 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:02.744321 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2eec5-a23a-4612-9053-c74c312c3f49 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:02.744620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2eec5-a23a-4612-9053-c74c312c3f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:02.758439 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 762/1544] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:45:02.761090 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53737c-df28-43bd-8410-070e7a11d99b None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:02.761736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53737c-df28-43bd-8410-070e7a11d99b None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:02.762294 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53737c-df28-43bd-8410-070e7a11d99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:02.777442 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-96f2eec5-a23a-4612-9053-c74c312c3f49 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:02.778695 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-96f2eec5-a23a-4612-9053-c74c312c3f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:02.799004 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-2f53737c-df28-43bd-8410-070e7a11d99b None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:02.803594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-2f53737c-df28-43bd-8410-070e7a11d99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:02.862246 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 783/1545] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:02.863073 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2eec5-a23a-4612-9053-c74c312c3f49 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:02.872930 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2eec5-a23a-4612-9053-c74c312c3f49 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:02.877804 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ac0b7-7493-477a-a2dc-3213bd00ec38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:02.877961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 763/1546] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:02.878468 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ac0b7-7493-477a-a2dc-3213bd00ec38 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:02.878755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ac0b7-7493-477a-a2dc-3213bd00ec38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:02.879132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ac0b7-7493-477a-a2dc-3213bd00ec38 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:02.879282 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382df8f-981a-43d8-b5b0-25b7a4059ad4 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:02.879575 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382df8f-981a-43d8-b5b0-25b7a4059ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:02.879834 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ac0b7-7493-477a-a2dc-3213bd00ec38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:02.879968 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382df8f-981a-43d8-b5b0-25b7a4059ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:02.881825 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ac0b7-7493-477a-a2dc-3213bd00ec38 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:02.898643 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-b382df8f-981a-43d8-b5b0-25b7a4059ad4 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:02.900189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-b382df8f-981a-43d8-b5b0-25b7a4059ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:02.908199 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 764/1547] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:02.909553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98528664-0c40-4f98-ad67-35e710c42eb8 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:02.909849 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98528664-0c40-4f98-ad67-35e710c42eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:02.910130 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98528664-0c40-4f98-ad67-35e710c42eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:02.927971 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-98528664-0c40-4f98-ad67-35e710c42eb8 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:02.929172 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-98528664-0c40-4f98-ad67-35e710c42eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:02.937907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 765/1548] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:02.941552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b51640-f3aa-46c2-a5b8-1181acdecd84 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:02.941552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b51640-f3aa-46c2-a5b8-1181acdecd84 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:02.941552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b51640-f3aa-46c2-a5b8-1181acdecd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:02.957826 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-51b51640-f3aa-46c2-a5b8-1181acdecd84 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:02.961699 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 784/1549] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:02 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:02.965810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-51b51640-f3aa-46c2-a5b8-1181acdecd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:02.965929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8dc9a-162e-4d12-8605-b1feb1ceac3d None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:02.965929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8dc9a-162e-4d12-8605-b1feb1ceac3d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:02.965929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8dc9a-162e-4d12-8605-b1feb1ceac3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:02.969507 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 766/1550] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:02.970815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4886cb-bde1-4df9-b2e2-01f54905c65d None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:02.971116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4886cb-bde1-4df9-b2e2-01f54905c65d None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:02.971579 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4886cb-bde1-4df9-b2e2-01f54905c65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:02.978859 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-6db8dc9a-162e-4d12-8605-b1feb1ceac3d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:02.980072 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6db8dc9a-162e-4d12-8605-b1feb1ceac3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:02.988349 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 785/1551] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:02.989667 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc055a63-9fa2-47b3-bf0e-95f82fde9136 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:02.989965 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc055a63-9fa2-47b3-bf0e-95f82fde9136 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:02.990091 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-6c4886cb-bde1-4df9-b2e2-01f54905c65d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:02.990270 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc055a63-9fa2-47b3-bf0e-95f82fde9136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:02.991242 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-6c4886cb-bde1-4df9-b2e2-01f54905c65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:03.001070 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 767/1552] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:03.002155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe9545-2caf-4e0e-9c47-507c8420d8ba None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.002444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe9545-2caf-4e0e-9c47-507c8420d8ba None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.002721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe9545-2caf-4e0e-9c47-507c8420d8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.009999 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-dc055a63-9fa2-47b3-bf0e-95f82fde9136 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:03.011215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-dc055a63-9fa2-47b3-bf0e-95f82fde9136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:03.019928 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 786/1553] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:03.021278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999abb9a-5318-44b1-b08d-93791653fb2a None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.021494 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-dbbe9545-2caf-4e0e-9c47-507c8420d8ba None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:03.021624 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999abb9a-5318-44b1-b08d-93791653fb2a None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.021850 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999abb9a-5318-44b1-b08d-93791653fb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.022652 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-dbbe9545-2caf-4e0e-9c47-507c8420d8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:03.032263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 768/1554] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:03.033100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe9545-2caf-4e0e-9c47-507c8420d8ba None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:03.040930 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-999abb9a-5318-44b1-b08d-93791653fb2a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:03.042206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-999abb9a-5318-44b1-b08d-93791653fb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:03.042918 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe9545-2caf-4e0e-9c47-507c8420d8ba None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:03.047394 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd932339-711e-4c85-95f3-768b9f5ab2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:03.048041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd932339-711e-4c85-95f3-768b9f5ab2de admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.048327 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd932339-711e-4c85-95f3-768b9f5ab2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.048641 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd932339-711e-4c85-95f3-768b9f5ab2de admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.049399 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd932339-711e-4c85-95f3-768b9f5ab2de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:45:03.049706 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd932339-711e-4c85-95f3-768b9f5ab2de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:03.050753 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: WARNING py.warnings [None req-fd932339-711e-4c85-95f3-768b9f5ab2de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:45:03.050753 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: warnings.warn(msg) Jun 23 22:45:03.050753 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: Jun 23 22:45:03.050983 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 787/1555] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:03.051630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd932339-711e-4c85-95f3-768b9f5ab2de admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:03.051895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999abb9a-5318-44b1-b08d-93791653fb2a None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:03.062435 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999abb9a-5318-44b1-b08d-93791653fb2a None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:03.070713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 769/1556] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:45:03.070713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341acae-21a2-4773-ae66-fea17bdfce01 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.070713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341acae-21a2-4773-ae66-fea17bdfce01 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.070713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341acae-21a2-4773-ae66-fea17bdfce01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.071189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a5b09-c90b-4e4d-b64d-0c1f8107503b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:03.071189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a5b09-c90b-4e4d-b64d-0c1f8107503b admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.071189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a5b09-c90b-4e4d-b64d-0c1f8107503b admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.071189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a5b09-c90b-4e4d-b64d-0c1f8107503b admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.071550 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a5b09-c90b-4e4d-b64d-0c1f8107503b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:03.071550 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a5b09-c90b-4e4d-b64d-0c1f8107503b admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:03.081960 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 788/1557] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:03.083200 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359560f0-ca44-4e05-9f96-67f378d16c1d None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.083502 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359560f0-ca44-4e05-9f96-67f378d16c1d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.083789 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359560f0-ca44-4e05-9f96-67f378d16c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.084458 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-6341acae-21a2-4773-ae66-fea17bdfce01 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:03.085752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-6341acae-21a2-4773-ae66-fea17bdfce01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:03.093761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 770/1558] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:03.094974 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc37fe6-79fc-4244-9981-95d0e416ca00 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.095267 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc37fe6-79fc-4244-9981-95d0e416ca00 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.095610 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc37fe6-79fc-4244-9981-95d0e416ca00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.101279 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-359560f0-ca44-4e05-9f96-67f378d16c1d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:03.102479 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-359560f0-ca44-4e05-9f96-67f378d16c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:03.111002 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 789/1559] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:03.111851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359560f0-ca44-4e05-9f96-67f378d16c1d None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:03.115418 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-bfc37fe6-79fc-4244-9981-95d0e416ca00 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:03.116597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-bfc37fe6-79fc-4244-9981-95d0e416ca00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:03.124006 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359560f0-ca44-4e05-9f96-67f378d16c1d None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:03.125549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 771/1560] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:03.126376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc37fe6-79fc-4244-9981-95d0e416ca00 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:03.129207 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d15e45-fca6-4ec2-8df4-efc95d2134ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:03.129807 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d15e45-fca6-4ec2-8df4-efc95d2134ba admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.130086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d15e45-fca6-4ec2-8df4-efc95d2134ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.130362 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d15e45-fca6-4ec2-8df4-efc95d2134ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.131143 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d15e45-fca6-4ec2-8df4-efc95d2134ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:45:03.131482 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d15e45-fca6-4ec2-8df4-efc95d2134ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:03.133102 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d15e45-fca6-4ec2-8df4-efc95d2134ba admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:03.135547 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc37fe6-79fc-4244-9981-95d0e416ca00 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:03.139937 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9d1f2-3778-4e67-8607-a21841a4c843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:03.140594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9d1f2-3778-4e67-8607-a21841a4c843 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.140995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9d1f2-3778-4e67-8607-a21841a4c843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.141280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9d1f2-3778-4e67-8607-a21841a4c843 admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.141998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9d1f2-3778-4e67-8607-a21841a4c843 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:45:03.142304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9d1f2-3778-4e67-8607-a21841a4c843 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:03.143868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9d1f2-3778-4e67-8607-a21841a4c843 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:03.144498 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 790/1561] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:45:03.145323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d15e45-fca6-4ec2-8df4-efc95d2134ba admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:03.160730 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 772/1562] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:45:03.160903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d15e45-fca6-4ec2-8df4-efc95d2134ba admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:03.161003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9d1f2-3778-4e67-8607-a21841a4c843 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:03.164395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b165e-3278-4e18-abee-e43570fdcda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:03.165080 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b165e-3278-4e18-abee-e43570fdcda2 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.165379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b165e-3278-4e18-abee-e43570fdcda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.165647 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b165e-3278-4e18-abee-e43570fdcda2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.166372 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b165e-3278-4e18-abee-e43570fdcda2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:45:03.166683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b165e-3278-4e18-abee-e43570fdcda2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:03.168261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b165e-3278-4e18-abee-e43570fdcda2 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:03.171333 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9d1f2-3778-4e67-8607-a21841a4c843 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:03.176047 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616872f-ec4d-4b38-a84a-df3912376f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:03.176791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616872f-ec4d-4b38-a84a-df3912376f57 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.177109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616872f-ec4d-4b38-a84a-df3912376f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.177491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616872f-ec4d-4b38-a84a-df3912376f57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.178188 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b616872f-ec4d-4b38-a84a-df3912376f57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:03.180171 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b616872f-ec4d-4b38-a84a-df3912376f57 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:03.188557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 791/1563] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:45:03.188557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b165e-3278-4e18-abee-e43570fdcda2 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:03.193471 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b165e-3278-4e18-abee-e43570fdcda2 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:03.198480 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744ea34-20f1-4d0c-bea3-a6d6daac93a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:03.199136 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744ea34-20f1-4d0c-bea3-a6d6daac93a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.199432 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744ea34-20f1-4d0c-bea3-a6d6daac93a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.199711 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744ea34-20f1-4d0c-bea3-a6d6daac93a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.200418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744ea34-20f1-4d0c-bea3-a6d6daac93a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:03.202036 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744ea34-20f1-4d0c-bea3-a6d6daac93a0 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:03.212087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 792/1564] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:03.213041 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744ea34-20f1-4d0c-bea3-a6d6daac93a0 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:03.222659 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744ea34-20f1-4d0c-bea3-a6d6daac93a0 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:03.227199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd96375-e7b2-4f2b-9d9a-0a227db87c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:03.227820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd96375-e7b2-4f2b-9d9a-0a227db87c4c admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.228108 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd96375-e7b2-4f2b-9d9a-0a227db87c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.228415 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd96375-e7b2-4f2b-9d9a-0a227db87c4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.229222 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd96375-e7b2-4f2b-9d9a-0a227db87c4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:45:03.229533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd96375-e7b2-4f2b-9d9a-0a227db87c4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:03.231618 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd96375-e7b2-4f2b-9d9a-0a227db87c4c admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:03.243177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 793/1565] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:45:03.243939 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd96375-e7b2-4f2b-9d9a-0a227db87c4c admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:03.253927 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd96375-e7b2-4f2b-9d9a-0a227db87c4c admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:03.258367 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78eb2d2-739a-40ac-8313-18d5cbc92929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:03.258982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eb2d2-739a-40ac-8313-18d5cbc92929 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.259295 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eb2d2-739a-40ac-8313-18d5cbc92929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.259579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eb2d2-739a-40ac-8313-18d5cbc92929 admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.260290 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78eb2d2-739a-40ac-8313-18d5cbc92929 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:45:03.260622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78eb2d2-739a-40ac-8313-18d5cbc92929 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:03.262164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78eb2d2-739a-40ac-8313-18d5cbc92929 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:03.274018 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:03.274212 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 794/1566] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:45:03.275331 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78eb2d2-739a-40ac-8313-18d5cbc92929 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:03.287769 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 773/1567] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:03 2022] POST /identity/v3/projects => generated 387 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:03.287769 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616872f-ec4d-4b38-a84a-df3912376f57 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:03.295643 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78eb2d2-739a-40ac-8313-18d5cbc92929 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:03.341580 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:03.346801 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028bd809-527f-4309-b511-76dcb2f56b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:03.347467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bd809-527f-4309-b511-76dcb2f56b7d admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.347751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bd809-527f-4309-b511-76dcb2f56b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.348030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bd809-527f-4309-b511-76dcb2f56b7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.348830 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028bd809-527f-4309-b511-76dcb2f56b7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:45:03.349121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028bd809-527f-4309-b511-76dcb2f56b7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:03.350745 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028bd809-527f-4309-b511-76dcb2f56b7d admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:03.351813 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616872f-ec4d-4b38-a84a-df3912376f57 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:03.357102 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9cdf8-f608-4309-8892-4512fccb968e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:03.357715 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9cdf8-f608-4309-8892-4512fccb968e admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.358003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9cdf8-f608-4309-8892-4512fccb968e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.358291 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9cdf8-f608-4309-8892-4512fccb968e admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.358979 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9cdf8-f608-4309-8892-4512fccb968e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:03.360902 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9cdf8-f608-4309-8892-4512fccb968e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:03.364670 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 795/1568] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:45:03.365253 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028bd809-527f-4309-b511-76dcb2f56b7d admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:03.376031 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028bd809-527f-4309-b511-76dcb2f56b7d admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:03.380554 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da13d1-d408-4469-beab-1d9f9f319dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:03.381228 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da13d1-d408-4469-beab-1d9f9f319dce admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.381551 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da13d1-d408-4469-beab-1d9f9f319dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.381816 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da13d1-d408-4469-beab-1d9f9f319dce admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.382497 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da13d1-d408-4469-beab-1d9f9f319dce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:03.384394 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da13d1-d408-4469-beab-1d9f9f319dce admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:03.439950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 774/1569] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:03 2022] POST /identity/v3/projects => generated 405 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:03.440803 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9cdf8-f608-4309-8892-4512fccb968e admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:03.480066 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 796/1570] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:03 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:03.481063 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da13d1-d408-4469-beab-1d9f9f319dce admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:03.506862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9cdf8-f608-4309-8892-4512fccb968e admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:03.543181 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da13d1-d408-4469-beab-1d9f9f319dce admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:03.548453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eea1a9-3f61-429a-a550-392e323e4736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:03.549377 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eea1a9-3f61-429a-a550-392e323e4736 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.549666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eea1a9-3f61-429a-a550-392e323e4736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.549946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eea1a9-3f61-429a-a550-392e323e4736 admin admin] PATH_INFO: `/v3/projects/fd707e85ae3142e7b2118b13a3587df5/users/18861c09a6d241fda06aefcfad0f8eb1/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.562215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76318c10-0a14-4075-9cc5-a9bf9f015d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:03.562878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76318c10-0a14-4075-9cc5-a9bf9f015d80 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.563162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76318c10-0a14-4075-9cc5-a9bf9f015d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.563443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76318c10-0a14-4075-9cc5-a9bf9f015d80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.564134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76318c10-0a14-4075-9cc5-a9bf9f015d80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:03.564430 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eea1a9-3f61-429a-a550-392e323e4736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd707e85ae3142e7b2118b13a3587df5, user_id=18861c09a6d241fda06aefcfad0f8eb1, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:03.565488 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: WARNING py.warnings [None req-43eea1a9-3f61-429a-a550-392e323e4736 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:45:03.565488 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: warnings.warn(msg) Jun 23 22:45:03.565488 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: Jun 23 22:45:03.566076 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76318c10-0a14-4075-9cc5-a9bf9f015d80 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:03.567411 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eea1a9-3f61-429a-a550-392e323e4736 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:03.637770 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 797/1571] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:03 2022] PUT /identity/v3/projects/fd707e85ae3142e7b2118b13a3587df5/users/18861c09a6d241fda06aefcfad0f8eb1/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:03.638503 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eea1a9-3f61-429a-a550-392e323e4736 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:03.674002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 775/1572] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:03 2022] POST /identity/v3/projects => generated 385 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:03.676896 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76318c10-0a14-4075-9cc5-a9bf9f015d80 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:03.698207 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:03.703559 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eea1a9-3f61-429a-a550-392e323e4736 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:03.736674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76318c10-0a14-4075-9cc5-a9bf9f015d80 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:03.741300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397727ca-f24b-4b39-8253-74ce216f6b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:03.741990 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397727ca-f24b-4b39-8253-74ce216f6b29 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.742386 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397727ca-f24b-4b39-8253-74ce216f6b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.749513 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397727ca-f24b-4b39-8253-74ce216f6b29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.749513 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397727ca-f24b-4b39-8253-74ce216f6b29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:03.749513 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397727ca-f24b-4b39-8253-74ce216f6b29 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:03.759883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:03.761438 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fa6cf-4580-46dc-bf31-183a181fd785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:03.762110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fa6cf-4580-46dc-bf31-183a181fd785 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.762433 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fa6cf-4580-46dc-bf31-183a181fd785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.762770 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fa6cf-4580-46dc-bf31-183a181fd785 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.763531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69fa6cf-4580-46dc-bf31-183a181fd785 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:03.765480 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69fa6cf-4580-46dc-bf31-183a181fd785 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:03.856802 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 776/1573] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:03 2022] POST /identity/v3/projects => generated 373 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:03.857721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397727ca-f24b-4b39-8253-74ce216f6b29 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:03.869207 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 798/1574] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:03 2022] POST /identity/v3/projects => generated 389 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:03.869207 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fa6cf-4580-46dc-bf31-183a181fd785 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:03.922146 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397727ca-f24b-4b39-8253-74ce216f6b29 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:03.933123 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fa6cf-4580-46dc-bf31-183a181fd785 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:03.937859 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281ff2b-0c3c-46ab-ace2-5cc4c21dd751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:03.938522 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281ff2b-0c3c-46ab-ace2-5cc4c21dd751 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.938809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281ff2b-0c3c-46ab-ace2-5cc4c21dd751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.939121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281ff2b-0c3c-46ab-ace2-5cc4c21dd751 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.939778 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7281ff2b-0c3c-46ab-ace2-5cc4c21dd751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:03.941709 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7281ff2b-0c3c-46ab-ace2-5cc4c21dd751 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:03.981221 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebb7d6-22aa-4158-8f74-77cdb88b7531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:03.981884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebb7d6-22aa-4158-8f74-77cdb88b7531 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:03.982167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebb7d6-22aa-4158-8f74-77cdb88b7531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:03.982441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebb7d6-22aa-4158-8f74-77cdb88b7531 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:03.983118 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebb7d6-22aa-4158-8f74-77cdb88b7531 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:03.984171 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: WARNING py.warnings [None req-d2ebb7d6-22aa-4158-8f74-77cdb88b7531 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:45:03.984171 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: warnings.warn(msg) Jun 23 22:45:03.984171 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: Jun 23 22:45:03.985395 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebb7d6-22aa-4158-8f74-77cdb88b7531 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.010844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 799/1575] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:03 2022] POST /identity/v3/projects => generated 365 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:04.011718 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281ff2b-0c3c-46ab-ace2-5cc4c21dd751 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.073421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 777/1576] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:03 2022] POST /identity/v3/users => generated 294 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:04.074232 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281ff2b-0c3c-46ab-ace2-5cc4c21dd751 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.074362 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebb7d6-22aa-4158-8f74-77cdb88b7531 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.080018 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2496b-010c-45e2-bd1a-63008f5ef4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.080705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2496b-010c-45e2-bd1a-63008f5ef4c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.081002 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2496b-010c-45e2-bd1a-63008f5ef4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.081286 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2496b-010c-45e2-bd1a-63008f5ef4c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.081979 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2496b-010c-45e2-bd1a-63008f5ef4c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.083965 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2496b-010c-45e2-bd1a-63008f5ef4c9 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.084767 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebb7d6-22aa-4158-8f74-77cdb88b7531 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.089385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecb374-24f0-48e8-a36e-411007aed567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.090040 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecb374-24f0-48e8-a36e-411007aed567 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.090366 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecb374-24f0-48e8-a36e-411007aed567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.091179 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecb374-24f0-48e8-a36e-411007aed567 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.092021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deecb374-24f0-48e8-a36e-411007aed567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.094078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deecb374-24f0-48e8-a36e-411007aed567 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.158589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 800/1577] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:04 2022] POST /identity/v3/users => generated 303 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:04.159939 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294d10d-e8d3-4189-9567-47678eb234b3 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.160239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294d10d-e8d3-4189-9567-47678eb234b3 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.160521 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294d10d-e8d3-4189-9567-47678eb234b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.165908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:04.173283 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 778/1578] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:04 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:04.173283 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecb374-24f0-48e8-a36e-411007aed567 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.180344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecb374-24f0-48e8-a36e-411007aed567 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.185134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d323a34-55cf-4ee7-9c45-889394c28611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.185786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d323a34-55cf-4ee7-9c45-889394c28611 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.186070 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d323a34-55cf-4ee7-9c45-889394c28611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.186348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d323a34-55cf-4ee7-9c45-889394c28611 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.187030 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d323a34-55cf-4ee7-9c45-889394c28611 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.187181 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-5294d10d-e8d3-4189-9567-47678eb234b3 None None] MFA Rules not processed for user `18861c09a6d241fda06aefcfad0f8eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:04.188416 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-5294d10d-e8d3-4189-9567-47678eb234b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:04.188977 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d323a34-55cf-4ee7-9c45-889394c28611 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.251841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 801/1579] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:04 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:04.252687 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5294d10d-e8d3-4189-9567-47678eb234b3 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.263050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 779/1580] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:04 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:04.263867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d323a34-55cf-4ee7-9c45-889394c28611 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.265438 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5294d10d-e8d3-4189-9567-47678eb234b3 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.270019 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b2b7b-a65d-4c73-b6bc-1a3012710648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.270692 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b2b7b-a65d-4c73-b6bc-1a3012710648 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.270973 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b2b7b-a65d-4c73-b6bc-1a3012710648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.271245 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b2b7b-a65d-4c73-b6bc-1a3012710648 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.271937 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b2b7b-a65d-4c73-b6bc-1a3012710648 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.273896 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b2b7b-a65d-4c73-b6bc-1a3012710648 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.275740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d323a34-55cf-4ee7-9c45-889394c28611 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.280308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d865ff5-73ff-4382-8a70-5d497f79a5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.281005 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d865ff5-73ff-4382-8a70-5d497f79a5b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.281294 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d865ff5-73ff-4382-8a70-5d497f79a5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.281575 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d865ff5-73ff-4382-8a70-5d497f79a5b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.282269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d865ff5-73ff-4382-8a70-5d497f79a5b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.286762 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d865ff5-73ff-4382-8a70-5d497f79a5b1 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.363570 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 802/1581] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:04 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:04.364404 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b2b7b-a65d-4c73-b6bc-1a3012710648 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.374891 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b2b7b-a65d-4c73-b6bc-1a3012710648 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.376337 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 780/1582] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:04 2022] POST /identity/v3/users => generated 295 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:04.377216 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d865ff5-73ff-4382-8a70-5d497f79a5b1 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.382479 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb9d4d-cd57-4572-b34c-936b58442477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.383139 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb9d4d-cd57-4572-b34c-936b58442477 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.383424 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb9d4d-cd57-4572-b34c-936b58442477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.383700 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb9d4d-cd57-4572-b34c-936b58442477 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.384377 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb9d4d-cd57-4572-b34c-936b58442477 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.386341 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb9d4d-cd57-4572-b34c-936b58442477 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.387233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d865ff5-73ff-4382-8a70-5d497f79a5b1 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.392187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff6211-7af0-42fd-93b2-7ec7d818b2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.392873 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff6211-7af0-42fd-93b2-7ec7d818b2a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.394948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff6211-7af0-42fd-93b2-7ec7d818b2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.394948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff6211-7af0-42fd-93b2-7ec7d818b2a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.394948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff6211-7af0-42fd-93b2-7ec7d818b2a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.395912 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff6211-7af0-42fd-93b2-7ec7d818b2a7 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.406346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 781/1583] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:04.407176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff6211-7af0-42fd-93b2-7ec7d818b2a7 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.416291 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff6211-7af0-42fd-93b2-7ec7d818b2a7 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.420864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fac1d-84ed-4ead-ac22-5c2113aa4c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.423816 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fac1d-84ed-4ead-ac22-5c2113aa4c0b admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.423816 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fac1d-84ed-4ead-ac22-5c2113aa4c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.423816 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fac1d-84ed-4ead-ac22-5c2113aa4c0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.423816 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751fac1d-84ed-4ead-ac22-5c2113aa4c0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.424320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751fac1d-84ed-4ead-ac22-5c2113aa4c0b admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.435014 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 782/1584] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:04.435852 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fac1d-84ed-4ead-ac22-5c2113aa4c0b admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.445072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fac1d-84ed-4ead-ac22-5c2113aa4c0b admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.449579 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c98568-b76a-4b1c-a7bd-c50db6e1d857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.452345 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c98568-b76a-4b1c-a7bd-c50db6e1d857 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.452345 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c98568-b76a-4b1c-a7bd-c50db6e1d857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.452345 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c98568-b76a-4b1c-a7bd-c50db6e1d857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.452345 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c98568-b76a-4b1c-a7bd-c50db6e1d857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.453607 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c98568-b76a-4b1c-a7bd-c50db6e1d857 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.463966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 783/1585] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:04.464843 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c98568-b76a-4b1c-a7bd-c50db6e1d857 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.473194 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:04.478721 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 803/1586] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:04 2022] POST /identity/v3/users => generated 283 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:04.479562 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb9d4d-cd57-4572-b34c-936b58442477 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.489491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb9d4d-cd57-4572-b34c-936b58442477 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.494255 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae385ec2-7b1b-4d26-9628-2009ecc3aece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.494911 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae385ec2-7b1b-4d26-9628-2009ecc3aece admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.495196 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae385ec2-7b1b-4d26-9628-2009ecc3aece admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.495472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae385ec2-7b1b-4d26-9628-2009ecc3aece admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.496195 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae385ec2-7b1b-4d26-9628-2009ecc3aece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.498684 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae385ec2-7b1b-4d26-9628-2009ecc3aece admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.509864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 804/1587] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:04.510711 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae385ec2-7b1b-4d26-9628-2009ecc3aece admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.520143 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c98568-b76a-4b1c-a7bd-c50db6e1d857 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.520664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae385ec2-7b1b-4d26-9628-2009ecc3aece admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.523854 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd5969-ea79-40b9-9d77-204ef60c379c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.524504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd5969-ea79-40b9-9d77-204ef60c379c service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.524881 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd5969-ea79-40b9-9d77-204ef60c379c service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.525148 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee0ff9-7882-4d3b-bae4-c857f65976fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.525272 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd5969-ea79-40b9-9d77-204ef60c379c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.525786 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee0ff9-7882-4d3b-bae4-c857f65976fe admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.526069 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee0ff9-7882-4d3b-bae4-c857f65976fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.526342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee0ff9-7882-4d3b-bae4-c857f65976fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.527093 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee0ff9-7882-4d3b-bae4-c857f65976fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.529042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee0ff9-7882-4d3b-bae4-c857f65976fe admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.534189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd5969-ea79-40b9-9d77-204ef60c379c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.535628 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd5969-ea79-40b9-9d77-204ef60c379c service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.539131 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 805/1588] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:04.539968 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee0ff9-7882-4d3b-bae4-c857f65976fe admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.546083 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 784/1589] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:04 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:04.546954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd5969-ea79-40b9-9d77-204ef60c379c service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.551878 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee0ff9-7882-4d3b-bae4-c857f65976fe admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.556122 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4c116-0caa-4c12-b1eb-d8e29dc4db59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.556870 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4c116-0caa-4c12-b1eb-d8e29dc4db59 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.557148 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4c116-0caa-4c12-b1eb-d8e29dc4db59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.557589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4c116-0caa-4c12-b1eb-d8e29dc4db59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.557727 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd5969-ea79-40b9-9d77-204ef60c379c service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.558191 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4c116-0caa-4c12-b1eb-d8e29dc4db59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.560306 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4c116-0caa-4c12-b1eb-d8e29dc4db59 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.563911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d988466-077a-4eb8-9721-e358d718e194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.565064 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d988466-077a-4eb8-9721-e358d718e194 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.566026 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d988466-077a-4eb8-9721-e358d718e194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.566308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d988466-077a-4eb8-9721-e358d718e194 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.567032 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d988466-077a-4eb8-9721-e358d718e194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.568642 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d988466-077a-4eb8-9721-e358d718e194 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.572465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 806/1590] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:04.573557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4c116-0caa-4c12-b1eb-d8e29dc4db59 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.580335 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 785/1591] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:04.581487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d988466-077a-4eb8-9721-e358d718e194 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.588531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4c116-0caa-4c12-b1eb-d8e29dc4db59 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.591167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d988466-077a-4eb8-9721-e358d718e194 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.593446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbfa74-f90c-448d-af72-013b60b2ac23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.594244 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbfa74-f90c-448d-af72-013b60b2ac23 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.594532 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbfa74-f90c-448d-af72-013b60b2ac23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.594812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbfa74-f90c-448d-af72-013b60b2ac23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.595539 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbfa74-f90c-448d-af72-013b60b2ac23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.596755 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee45be6-2e43-4400-8b09-7c8e783ae109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.597310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbfa74-f90c-448d-af72-013b60b2ac23 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.597418 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee45be6-2e43-4400-8b09-7c8e783ae109 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.597680 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee45be6-2e43-4400-8b09-7c8e783ae109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.597959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee45be6-2e43-4400-8b09-7c8e783ae109 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.598864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee45be6-2e43-4400-8b09-7c8e783ae109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.600861 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee45be6-2e43-4400-8b09-7c8e783ae109 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.611622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 807/1592] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:04.611622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbfa74-f90c-448d-af72-013b60b2ac23 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.612756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 786/1593] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:04.616755 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee45be6-2e43-4400-8b09-7c8e783ae109 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.622605 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbfa74-f90c-448d-af72-013b60b2ac23 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.628387 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee45be6-2e43-4400-8b09-7c8e783ae109 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.628784 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d80896-ca81-404f-b757-390021d8c44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.629652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d80896-ca81-404f-b757-390021d8c44c admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.630071 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d80896-ca81-404f-b757-390021d8c44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.630695 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d80896-ca81-404f-b757-390021d8c44c admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.631701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d80896-ca81-404f-b757-390021d8c44c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.633473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d80896-ca81-404f-b757-390021d8c44c admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.634926 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e241ee-7c17-44cf-9af9-b314eb52625d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.635715 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e241ee-7c17-44cf-9af9-b314eb52625d admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.636003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e241ee-7c17-44cf-9af9-b314eb52625d admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.636279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e241ee-7c17-44cf-9af9-b314eb52625d admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.637037 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e241ee-7c17-44cf-9af9-b314eb52625d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.638660 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e241ee-7c17-44cf-9af9-b314eb52625d admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.647422 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 808/1594] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:04.648151 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d80896-ca81-404f-b757-390021d8c44c admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.653283 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 787/1595] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:04.654212 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e241ee-7c17-44cf-9af9-b314eb52625d admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.658839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d80896-ca81-404f-b757-390021d8c44c admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.663782 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ea991-2489-4338-b3fc-4940ea680ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.663947 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e241ee-7c17-44cf-9af9-b314eb52625d admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.664647 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ea991-2489-4338-b3fc-4940ea680ccc admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.664941 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ea991-2489-4338-b3fc-4940ea680ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.665276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ea991-2489-4338-b3fc-4940ea680ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.666015 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ea991-2489-4338-b3fc-4940ea680ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.667841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ea991-2489-4338-b3fc-4940ea680ccc admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.669386 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a830c-f880-481a-8b3f-bf1c64721453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.670129 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a830c-f880-481a-8b3f-bf1c64721453 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.670425 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a830c-f880-481a-8b3f-bf1c64721453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.670692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a830c-f880-481a-8b3f-bf1c64721453 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.671416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a830c-f880-481a-8b3f-bf1c64721453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.673034 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a830c-f880-481a-8b3f-bf1c64721453 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.680439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 809/1596] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:04.681353 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ea991-2489-4338-b3fc-4940ea680ccc admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.688066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 788/1597] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:04.689267 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a830c-f880-481a-8b3f-bf1c64721453 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.693654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ea991-2489-4338-b3fc-4940ea680ccc admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.699859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a830c-f880-481a-8b3f-bf1c64721453 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.700918 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60d39d-3a00-4cc8-9ec0-b11b3ea60cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.701967 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60d39d-3a00-4cc8-9ec0-b11b3ea60cd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.702375 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60d39d-3a00-4cc8-9ec0-b11b3ea60cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.702714 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60d39d-3a00-4cc8-9ec0-b11b3ea60cd0 admin admin] PATH_INFO: `/v3/projects/13b00f92bcc449daadba8838f4c87a62/users/5107e87ee4c34ef9ac96fa4a3a899bda/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.704664 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e5125-1f72-4a4a-bd0d-7fe0a9c1b9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.705545 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e5125-1f72-4a4a-bd0d-7fe0a9c1b9a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.705828 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e5125-1f72-4a4a-bd0d-7fe0a9c1b9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.706103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e5125-1f72-4a4a-bd0d-7fe0a9c1b9a8 admin admin] PATH_INFO: `/v3/projects/9109b68ecbde446ea5fe39f1b4b8df11/users/ff743b2ec8db4bc481652a6c8110407b/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.718223 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60d39d-3a00-4cc8-9ec0-b11b3ea60cd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13b00f92bcc449daadba8838f4c87a62, user_id=5107e87ee4c34ef9ac96fa4a3a899bda, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.720904 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60d39d-3a00-4cc8-9ec0-b11b3ea60cd0 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.722326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e5125-1f72-4a4a-bd0d-7fe0a9c1b9a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9109b68ecbde446ea5fe39f1b4b8df11, user_id=ff743b2ec8db4bc481652a6c8110407b, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.725325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e5125-1f72-4a4a-bd0d-7fe0a9c1b9a8 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.781377 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 810/1598] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:04 2022] PUT /identity/v3/projects/13b00f92bcc449daadba8838f4c87a62/users/5107e87ee4c34ef9ac96fa4a3a899bda/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:04.782269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60d39d-3a00-4cc8-9ec0-b11b3ea60cd0 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.807349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 789/1599] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:04 2022] PUT /identity/v3/projects/9109b68ecbde446ea5fe39f1b4b8df11/users/ff743b2ec8db4bc481652a6c8110407b/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:04.812981 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e5125-1f72-4a4a-bd0d-7fe0a9c1b9a8 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.817579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:04.850233 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60d39d-3a00-4cc8-9ec0-b11b3ea60cd0 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.877673 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e5125-1f72-4a4a-bd0d-7fe0a9c1b9a8 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.882556 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb4fdf-17da-482c-a01c-dbc383387d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.886636 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb4fdf-17da-482c-a01c-dbc383387d05 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.886636 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb4fdf-17da-482c-a01c-dbc383387d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.886636 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb4fdf-17da-482c-a01c-dbc383387d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.886636 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb4fdf-17da-482c-a01c-dbc383387d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.886636 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb4fdf-17da-482c-a01c-dbc383387d05 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.896860 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 790/1600] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:04.897840 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb4fdf-17da-482c-a01c-dbc383387d05 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:04.907906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb4fdf-17da-482c-a01c-dbc383387d05 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:04.908632 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ae300-9986-464e-b632-a8dbfd3c7ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.909570 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ae300-9986-464e-b632-a8dbfd3c7ec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.909809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ae300-9986-464e-b632-a8dbfd3c7ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.910187 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ae300-9986-464e-b632-a8dbfd3c7ec5 admin admin] PATH_INFO: `/v3/projects/a35daf906b2e442c9b27cb6f83673f0a/users/365b889f8f1a4c5dad09364e5df705ba/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.913199 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd3433-95d2-4b32-ad07-6d45b7f71419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:04.915548 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd3433-95d2-4b32-ad07-6d45b7f71419 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:04.915548 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd3433-95d2-4b32-ad07-6d45b7f71419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:04.915548 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd3433-95d2-4b32-ad07-6d45b7f71419 admin admin] PATH_INFO: `/v3/projects/18ed82f19aa2407dac94c2dc67a3fa9c/users/1d7e431402f8433fb7565f69103485ce/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:04.927772 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ae300-9986-464e-b632-a8dbfd3c7ec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a35daf906b2e442c9b27cb6f83673f0a, user_id=365b889f8f1a4c5dad09364e5df705ba, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.930974 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ae300-9986-464e-b632-a8dbfd3c7ec5 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:04.932489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd3433-95d2-4b32-ad07-6d45b7f71419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18ed82f19aa2407dac94c2dc67a3fa9c, user_id=1d7e431402f8433fb7565f69103485ce, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:04.935987 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd3433-95d2-4b32-ad07-6d45b7f71419 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:05.045755 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 791/1602] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:04 2022] PUT /identity/v3/projects/18ed82f19aa2407dac94c2dc67a3fa9c/users/1d7e431402f8433fb7565f69103485ce/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:05.045960 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 811/1602] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:04 2022] PUT /identity/v3/projects/a35daf906b2e442c9b27cb6f83673f0a/users/365b889f8f1a4c5dad09364e5df705ba/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:05.046587 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ae300-9986-464e-b632-a8dbfd3c7ec5 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:05.047029 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd3433-95d2-4b32-ad07-6d45b7f71419 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:05.071502 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:05.117123 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ae300-9986-464e-b632-a8dbfd3c7ec5 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:05.121370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd3433-95d2-4b32-ad07-6d45b7f71419 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:05.124397 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf502df-bc96-4a9b-a946-40d96956cff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:05.124397 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf502df-bc96-4a9b-a946-40d96956cff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:05.124397 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf502df-bc96-4a9b-a946-40d96956cff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:05.124397 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf502df-bc96-4a9b-a946-40d96956cff8 admin admin] PATH_INFO: `/v3/projects/ecc700154c034f328172bcdc1487b88c/users/a53d7e0b52a84ce5966243f025277da2/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:05.126913 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5093acd-a38a-4eca-bc41-5ccad54efcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:05.127822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5093acd-a38a-4eca-bc41-5ccad54efcd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:05.128099 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5093acd-a38a-4eca-bc41-5ccad54efcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:05.128377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5093acd-a38a-4eca-bc41-5ccad54efcd8 admin admin] PATH_INFO: `/v3/projects/19ed20d55acb41fcb18006919476bf80/users/675d117397da452faa5221af0906fb12/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:05.139517 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf502df-bc96-4a9b-a946-40d96956cff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecc700154c034f328172bcdc1487b88c, user_id=a53d7e0b52a84ce5966243f025277da2, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:05.142243 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf502df-bc96-4a9b-a946-40d96956cff8 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:05.145520 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5093acd-a38a-4eca-bc41-5ccad54efcd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ed20d55acb41fcb18006919476bf80, user_id=675d117397da452faa5221af0906fb12, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:05.148207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5093acd-a38a-4eca-bc41-5ccad54efcd8 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:05.198022 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 812/1603] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:05 2022] PUT /identity/v3/projects/ecc700154c034f328172bcdc1487b88c/users/a53d7e0b52a84ce5966243f025277da2/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:05.199308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfad6d-93ff-48ce-8b60-7b4fb77f7d67 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:05.199634 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfad6d-93ff-48ce-8b60-7b4fb77f7d67 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:05.199955 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfad6d-93ff-48ce-8b60-7b4fb77f7d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:05.217118 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 792/1604] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:05 2022] PUT /identity/v3/projects/19ed20d55acb41fcb18006919476bf80/users/675d117397da452faa5221af0906fb12/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:05.218220 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7a1e0-82f8-4a77-bbb3-e2d214eda38f None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:05.218507 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7a1e0-82f8-4a77-bbb3-e2d214eda38f None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:05.218891 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7a1e0-82f8-4a77-bbb3-e2d214eda38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:05.220414 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-85dfad6d-93ff-48ce-8b60-7b4fb77f7d67 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:05.222371 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-85dfad6d-93ff-48ce-8b60-7b4fb77f7d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:05.237015 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-37c7a1e0-82f8-4a77-bbb3-e2d214eda38f None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:05.238208 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-37c7a1e0-82f8-4a77-bbb3-e2d214eda38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:05.259527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:05.287060 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 813/1605] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:05.287060 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfad6d-93ff-48ce-8b60-7b4fb77f7d67 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:05.298528 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfad6d-93ff-48ce-8b60-7b4fb77f7d67 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:05.305178 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1e4fd-7353-42b1-980a-6636a953918e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:05.306109 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1e4fd-7353-42b1-980a-6636a953918e admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:05.306466 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1e4fd-7353-42b1-980a-6636a953918e admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:05.306833 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1e4fd-7353-42b1-980a-6636a953918e admin admin] PATH_INFO: `/v3/projects/7bea8c4eba084af1a650862b5fa35276/users/44dd40d10d5846378771bd2307b952fe/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:05.320847 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 793/1606] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:05.328791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3b66a-be8b-4587-942d-7617c627e577 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:05.329257 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3b66a-be8b-4587-942d-7617c627e577 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:05.329797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3b66a-be8b-4587-942d-7617c627e577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:05.338455 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d1e4fd-7353-42b1-980a-6636a953918e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bea8c4eba084af1a650862b5fa35276, user_id=44dd40d10d5846378771bd2307b952fe, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:05.348091 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d1e4fd-7353-42b1-980a-6636a953918e admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:05.354943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-e5b3b66a-be8b-4587-942d-7617c627e577 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:05.356292 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e5b3b66a-be8b-4587-942d-7617c627e577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:05.364362 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 794/1607] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:05.365723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285612ca-d7c9-43f5-81b9-9b1f910a83b8 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:05.366081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285612ca-d7c9-43f5-81b9-9b1f910a83b8 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:05.366430 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285612ca-d7c9-43f5-81b9-9b1f910a83b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:05.384758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-285612ca-d7c9-43f5-81b9-9b1f910a83b8 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:05.386054 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-285612ca-d7c9-43f5-81b9-9b1f910a83b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:05.394596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 795/1608] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:05.395981 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f66e33-bd05-4644-9e27-04186041de9a None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:05.396125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f66e33-bd05-4644-9e27-04186041de9a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:05.396454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f66e33-bd05-4644-9e27-04186041de9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:05.427842 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-39f66e33-bd05-4644-9e27-04186041de9a None None] MFA Rules not processed for user `a53d7e0b52a84ce5966243f025277da2`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:05.429080 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-39f66e33-bd05-4644-9e27-04186041de9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:05.431091 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 814/1609] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:05 2022] PUT /identity/v3/projects/7bea8c4eba084af1a650862b5fa35276/users/44dd40d10d5846378771bd2307b952fe/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:05.432332 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1bbb9a-a374-405e-9cfa-65bcb5aa4ecc None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:05.432720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1bbb9a-a374-405e-9cfa-65bcb5aa4ecc None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:05.433067 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1bbb9a-a374-405e-9cfa-65bcb5aa4ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:05.452958 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-9d1bbb9a-a374-405e-9cfa-65bcb5aa4ecc None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:05.454223 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-9d1bbb9a-a374-405e-9cfa-65bcb5aa4ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:05.511729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 796/1610] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:05.513341 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f66e33-bd05-4644-9e27-04186041de9a None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:05.521912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 815/1611] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:05.526182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1bbb9a-a374-405e-9cfa-65bcb5aa4ecc None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:05.527496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-9d1bbb9a-a374-405e-9cfa-65bcb5aa4ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:05.557009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:05.592643 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1bbb9a-a374-405e-9cfa-65bcb5aa4ecc None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:05.596467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304e435-d25d-40a9-a475-c378e8935587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:05.597203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304e435-d25d-40a9-a475-c378e8935587 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:05.597498 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304e435-d25d-40a9-a475-c378e8935587 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:05.597881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304e435-d25d-40a9-a475-c378e8935587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:05.605597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f66e33-bd05-4644-9e27-04186041de9a None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:05.607936 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304e435-d25d-40a9-a475-c378e8935587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:05.609394 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304e435-d25d-40a9-a475-c378e8935587 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:05.609991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa3385-0dda-4545-8651-7520a2cdcbc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:05.610835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa3385-0dda-4545-8651-7520a2cdcbc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:05.610950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa3385-0dda-4545-8651-7520a2cdcbc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:05.611226 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa3385-0dda-4545-8651-7520a2cdcbc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:05.619111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 816/1612] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:05.620032 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304e435-d25d-40a9-a475-c378e8935587 service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:05.621281 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa3385-0dda-4545-8651-7520a2cdcbc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:05.623406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa3385-0dda-4545-8651-7520a2cdcbc5 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:05.630087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304e435-d25d-40a9-a475-c378e8935587 service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:05.634410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740e781-ee57-4592-a01b-0ee2a2843810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:05.635057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740e781-ee57-4592-a01b-0ee2a2843810 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:05.635344 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740e781-ee57-4592-a01b-0ee2a2843810 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:05.636011 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740e781-ee57-4592-a01b-0ee2a2843810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:05.651267 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740e781-ee57-4592-a01b-0ee2a2843810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:05.651267 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740e781-ee57-4592-a01b-0ee2a2843810 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:05.652820 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 797/1613] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:05.653881 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa3385-0dda-4545-8651-7520a2cdcbc5 service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:05.661073 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 817/1614] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:05.662528 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a08bd5-8638-4c70-be9d-193ffdcbfa45 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:05.662975 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a08bd5-8638-4c70-be9d-193ffdcbfa45 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:05.663336 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a08bd5-8638-4c70-be9d-193ffdcbfa45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:05.663651 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa3385-0dda-4545-8651-7520a2cdcbc5 service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:05.667671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f594b3-9dc5-4a52-bd10-37f0740f9fec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:05.672487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f594b3-9dc5-4a52-bd10-37f0740f9fec service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:05.672487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f594b3-9dc5-4a52-bd10-37f0740f9fec service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:05.672487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f594b3-9dc5-4a52-bd10-37f0740f9fec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:05.680023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f594b3-9dc5-4a52-bd10-37f0740f9fec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:05.681517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f594b3-9dc5-4a52-bd10-37f0740f9fec service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:05.693699 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-b1a08bd5-8638-4c70-be9d-193ffdcbfa45 None None] MFA Rules not processed for user `44dd40d10d5846378771bd2307b952fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:05.694995 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-b1a08bd5-8638-4c70-be9d-193ffdcbfa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:05.697090 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 798/1615] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:05.697954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f594b3-9dc5-4a52-bd10-37f0740f9fec service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:05.706308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f594b3-9dc5-4a52-bd10-37f0740f9fec service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:05.710690 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64e7f1-7360-4eeb-8dde-ed78f7d955f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:05.710931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64e7f1-7360-4eeb-8dde-ed78f7d955f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:05.711184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64e7f1-7360-4eeb-8dde-ed78f7d955f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:05.711468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64e7f1-7360-4eeb-8dde-ed78f7d955f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:05.722088 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64e7f1-7360-4eeb-8dde-ed78f7d955f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:05.723538 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64e7f1-7360-4eeb-8dde-ed78f7d955f6 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:05.734384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 799/1616] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:05.735677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7474ea7-7968-4e49-9291-c89b210ca1c2 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:05.735973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7474ea7-7968-4e49-9291-c89b210ca1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:05.736259 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7474ea7-7968-4e49-9291-c89b210ca1c2 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:05.736380 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:05.737582 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 800/1617] 10.222.0.33 () {54 vars in 961 bytes} [Thu Jun 23 22:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:45:05.744832 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f5d9e-4626-4e6b-b4d1-d989577b125a None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:05.745125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f5d9e-4626-4e6b-b4d1-d989577b125a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:05.745401 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f5d9e-4626-4e6b-b4d1-d989577b125a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:05.769835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-e11f5d9e-4626-4e6b-b4d1-d989577b125a None None] MFA Rules not processed for user `159cc8befc3a4d57bbd313832084db62`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:05.771001 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e11f5d9e-4626-4e6b-b4d1-d989577b125a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:05.792276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 818/1618] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:05.795713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a08bd5-8638-4c70-be9d-193ffdcbfa45 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:05.818881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a08bd5-8638-4c70-be9d-193ffdcbfa45 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:05.829254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44332d1b-a619-4c47-8707-0bd5c0382df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:05.830498 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44332d1b-a619-4c47-8707-0bd5c0382df4 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:05.831242 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44332d1b-a619-4c47-8707-0bd5c0382df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:05.831946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44332d1b-a619-4c47-8707-0bd5c0382df4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:05.833094 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44332d1b-a619-4c47-8707-0bd5c0382df4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:05.836867 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44332d1b-a619-4c47-8707-0bd5c0382df4 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:05.853304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 801/1619] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Jun 23 22:45:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:45:05.863074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f5d9e-4626-4e6b-b4d1-d989577b125a None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:05.873560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f5d9e-4626-4e6b-b4d1-d989577b125a None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:05.879407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd185b0-6cf5-4470-8464-2e713869d3df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:05.889302 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd185b0-6cf5-4470-8464-2e713869d3df service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:05.889302 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd185b0-6cf5-4470-8464-2e713869d3df service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:05.889302 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd185b0-6cf5-4470-8464-2e713869d3df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:05.889302 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd185b0-6cf5-4470-8464-2e713869d3df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:05.890284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd185b0-6cf5-4470-8464-2e713869d3df service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:05.899914 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 802/1620] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:05.947705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 819/1621] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:05 2022] POST /identity/v3/projects => generated 361 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:45:05.949344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd185b0-6cf5-4470-8464-2e713869d3df service nova] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:05.967930 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd185b0-6cf5-4470-8464-2e713869d3df service nova] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:06.062392 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a38bb5-a0fe-4a30-9556-9b28698f9c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:06.062864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a38bb5-a0fe-4a30-9556-9b28698f9c6f admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:06.063296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a38bb5-a0fe-4a30-9556-9b28698f9c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:06.063752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a38bb5-a0fe-4a30-9556-9b28698f9c6f admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:06.064614 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a38bb5-a0fe-4a30-9556-9b28698f9c6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:06.066655 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a38bb5-a0fe-4a30-9556-9b28698f9c6f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:06.184924 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e5a42-5c0b-416a-ae59-784c9e6fb6fe None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:06.185989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e5a42-5c0b-416a-ae59-784c9e6fb6fe None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:06.186494 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e5a42-5c0b-416a-ae59-784c9e6fb6fe None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:06.188487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:06.190959 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 820/1622] 10.222.0.33 () {54 vars in 912 bytes} [Thu Jun 23 22:45:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:45:06.196022 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 803/1623] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:05 2022] POST /identity/v3/users => generated 287 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:06.200213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e5a42-5c0b-416a-ae59-784c9e6fb6fe None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:06.204390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14f4b3-3d57-4a30-982c-81f0b7b8558c None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:06.205826 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14f4b3-3d57-4a30-982c-81f0b7b8558c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:06.207850 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14f4b3-3d57-4a30-982c-81f0b7b8558c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:06.233727 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e5a42-5c0b-416a-ae59-784c9e6fb6fe None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:06.242688 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-0c14f4b3-3d57-4a30-982c-81f0b7b8558c None None] MFA Rules not processed for user `159cc8befc3a4d57bbd313832084db62`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:06.243964 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-0c14f4b3-3d57-4a30-982c-81f0b7b8558c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:06.245245 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca3609-c8a6-45a4-a523-87325ced6fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:06.245937 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca3609-c8a6-45a4-a523-87325ced6fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:06.246229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca3609-c8a6-45a4-a523-87325ced6fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:06.246548 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca3609-c8a6-45a4-a523-87325ced6fe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:06.247296 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca3609-c8a6-45a4-a523-87325ced6fe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:06.249436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca3609-c8a6-45a4-a523-87325ced6fe3 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:06.299789 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 821/1624] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:45:06.304549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca3609-c8a6-45a4-a523-87325ced6fe3 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:06.324251 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca3609-c8a6-45a4-a523-87325ced6fe3 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:06.330531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c1335-4d48-430d-9122-37b47eefa48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:06.331573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c1335-4d48-430d-9122-37b47eefa48b admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:06.331865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c1335-4d48-430d-9122-37b47eefa48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:06.332229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c1335-4d48-430d-9122-37b47eefa48b admin admin] PATH_INFO: `/v3/projects/5f1871c73f0a44b18154023a2c3e733f/users/1b39eb26fb9644edae3333eba3f9b4f5/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:06.354741 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c1335-4d48-430d-9122-37b47eefa48b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f1871c73f0a44b18154023a2c3e733f, user_id=1b39eb26fb9644edae3333eba3f9b4f5, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:06.358100 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c1335-4d48-430d-9122-37b47eefa48b admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:06.386525 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 804/1625] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 22:45:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:06.407280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14f4b3-3d57-4a30-982c-81f0b7b8558c None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:06.456118 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 822/1626] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:06 2022] PUT /identity/v3/projects/5f1871c73f0a44b18154023a2c3e733f/users/1b39eb26fb9644edae3333eba3f9b4f5/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:06.458245 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c1335-4d48-430d-9122-37b47eefa48b admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:06.504957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14f4b3-3d57-4a30-982c-81f0b7b8558c None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:06.552202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c1335-4d48-430d-9122-37b47eefa48b admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:06.557546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868746c5-54dd-4a4a-be22-1ef5057eb29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:06.558395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868746c5-54dd-4a4a-be22-1ef5057eb29a admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:06.559752 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868746c5-54dd-4a4a-be22-1ef5057eb29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:06.559880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868746c5-54dd-4a4a-be22-1ef5057eb29a admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:06.561286 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868746c5-54dd-4a4a-be22-1ef5057eb29a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:06.564533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868746c5-54dd-4a4a-be22-1ef5057eb29a admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:06.576924 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 823/1627] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:06.579963 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868746c5-54dd-4a4a-be22-1ef5057eb29a admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:06.582596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cc160-1d80-4440-94ce-2a6d71a2afff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:06.583545 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cc160-1d80-4440-94ce-2a6d71a2afff service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:06.584562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cc160-1d80-4440-94ce-2a6d71a2afff service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:06.584562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cc160-1d80-4440-94ce-2a6d71a2afff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:06.590208 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868746c5-54dd-4a4a-be22-1ef5057eb29a admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:06.602846 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7da8d-40d3-4089-b937-7942549298e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:06.602846 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7da8d-40d3-4089-b937-7942549298e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:06.603089 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7da8d-40d3-4089-b937-7942549298e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:06.603211 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7da8d-40d3-4089-b937-7942549298e0 admin admin] PATH_INFO: `/v3/projects/5f1871c73f0a44b18154023a2c3e733f/users/1b39eb26fb9644edae3333eba3f9b4f5/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:06.606454 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb7da8d-40d3-4089-b937-7942549298e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f1871c73f0a44b18154023a2c3e733f, user_id=1b39eb26fb9644edae3333eba3f9b4f5, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:06.610849 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb7da8d-40d3-4089-b937-7942549298e0 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:06.628520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 824/1628] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:06 2022] PUT /identity/v3/projects/5f1871c73f0a44b18154023a2c3e733f/users/1b39eb26fb9644edae3333eba3f9b4f5/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:45:06.640843 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba15b58-4572-4e13-a7eb-6ce42d19e570 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:06.641220 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba15b58-4572-4e13-a7eb-6ce42d19e570 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:06.641715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba15b58-4572-4e13-a7eb-6ce42d19e570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:06.664290 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:06.688883 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-aba15b58-4572-4e13-a7eb-6ce42d19e570 None None] MFA Rules not processed for user `1b39eb26fb9644edae3333eba3f9b4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:06.691149 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-aba15b58-4572-4e13-a7eb-6ce42d19e570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:06.698829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cc160-1d80-4440-94ce-2a6d71a2afff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:06.702948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cc160-1d80-4440-94ce-2a6d71a2afff service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:06.725424 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 805/1629] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:45:06.783904 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:06.820895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 825/1630] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:06 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:06.833084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cc160-1d80-4440-94ce-2a6d71a2afff service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:06.842281 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cc160-1d80-4440-94ce-2a6d71a2afff service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:06.927010 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b44f6-a5b4-44ef-94fc-55f41c7320c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:06.927808 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b44f6-a5b4-44ef-94fc-55f41c7320c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:06.928112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b44f6-a5b4-44ef-94fc-55f41c7320c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:06.928372 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b44f6-a5b4-44ef-94fc-55f41c7320c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:06.943419 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b44f6-a5b4-44ef-94fc-55f41c7320c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:06.946022 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b44f6-a5b4-44ef-94fc-55f41c7320c5 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:06.959546 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 806/1631] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:06 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:07.302865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba15b58-4572-4e13-a7eb-6ce42d19e570 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:07.323549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba15b58-4572-4e13-a7eb-6ce42d19e570 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:07.333162 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bb45e-d482-47c0-a071-fc98bf63c9cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:07.336751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bb45e-d482-47c0-a071-fc98bf63c9cb service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:07.336751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bb45e-d482-47c0-a071-fc98bf63c9cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:07.336751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bb45e-d482-47c0-a071-fc98bf63c9cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:07.448421 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bb45e-d482-47c0-a071-fc98bf63c9cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:07.455953 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bb45e-d482-47c0-a071-fc98bf63c9cb service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:07.460271 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb637b0-d937-4be0-b1fd-09e1946643f1 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:07.460568 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb637b0-d937-4be0-b1fd-09e1946643f1 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:07.460873 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb637b0-d937-4be0-b1fd-09e1946643f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:07.463519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 826/1632] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:07.513980 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-3eb637b0-d937-4be0-b1fd-09e1946643f1 None None] MFA Rules not processed for user `ff743b2ec8db4bc481652a6c8110407b`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:07.515221 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3eb637b0-d937-4be0-b1fd-09e1946643f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:07.703952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 807/1633] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:07 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:07.710403 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bb45e-d482-47c0-a071-fc98bf63c9cb service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:07.837266 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bb45e-d482-47c0-a071-fc98bf63c9cb service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:07.842697 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd944f-7592-4b3c-81e8-5a2b1ab52ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:07.843438 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd944f-7592-4b3c-81e8-5a2b1ab52ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:07.843873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd944f-7592-4b3c-81e8-5a2b1ab52ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:07.844258 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd944f-7592-4b3c-81e8-5a2b1ab52ad0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:07.845136 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd944f-7592-4b3c-81e8-5a2b1ab52ad0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:07.847221 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd944f-7592-4b3c-81e8-5a2b1ab52ad0 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:07.945335 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb637b0-d937-4be0-b1fd-09e1946643f1 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:07.945584 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 827/1634] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:07 2022] POST /identity/v3/projects => generated 405 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:07.960862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb637b0-d937-4be0-b1fd-09e1946643f1 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:08.040312 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b95fb-fec6-415a-be08-a1c958cd2fad None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:08.040537 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b95fb-fec6-415a-be08-a1c958cd2fad None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:08.040839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b95fb-fec6-415a-be08-a1c958cd2fad None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:08.043071 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 828/1635] 10.222.0.33 () {54 vars in 961 bytes} [Thu Jun 23 22:45:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:45:08.057548 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b0415-bd8a-4752-81b7-de279f6a76e6 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:08.057548 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b0415-bd8a-4752-81b7-de279f6a76e6 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:08.057548 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b0415-bd8a-4752-81b7-de279f6a76e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:08.091095 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-9d0b0415-bd8a-4752-81b7-de279f6a76e6 None None] MFA Rules not processed for user `159cc8befc3a4d57bbd313832084db62`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:08.092346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-9d0b0415-bd8a-4752-81b7-de279f6a76e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:08.146901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85995b63-874d-4316-b535-9523bdc637b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:08.147088 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85995b63-874d-4316-b535-9523bdc637b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:08.147377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85995b63-874d-4316-b535-9523bdc637b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:08.147657 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85995b63-874d-4316-b535-9523bdc637b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:08.152037 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85995b63-874d-4316-b535-9523bdc637b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:08.152037 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85995b63-874d-4316-b535-9523bdc637b9 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:08.214097 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 829/1636] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Jun 23 22:45:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:45:08.223023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9b74e-9c4b-4a54-a6d3-2085c877263f None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:08.223440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9b74e-9c4b-4a54-a6d3-2085c877263f None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:08.223834 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9b74e-9c4b-4a54-a6d3-2085c877263f None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:08.226739 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 830/1637] 10.222.0.33 () {54 vars in 912 bytes} [Thu Jun 23 22:45:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:45:08.233406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d4b1f-d938-421b-b4ad-3ee7926f73ef None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:08.233924 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d4b1f-d938-421b-b4ad-3ee7926f73ef None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:08.234424 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d4b1f-d938-421b-b4ad-3ee7926f73ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:08.243201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:08.256987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-c08d4b1f-d938-421b-b4ad-3ee7926f73ef None None] MFA Rules not processed for user `159cc8befc3a4d57bbd313832084db62`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:08.258221 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-c08d4b1f-d938-421b-b4ad-3ee7926f73ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:08.264607 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 808/1638] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:07 2022] POST /identity/v3/users => generated 309 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:08.265559 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85995b63-874d-4316-b535-9523bdc637b9 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:08.280224 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 831/1639] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 22:45:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:45:08.285751 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85995b63-874d-4316-b535-9523bdc637b9 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:08.294240 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d4b1f-d938-421b-b4ad-3ee7926f73ef None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:08.298423 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0297bc47-fd57-45b7-b8b6-1498cc6bef65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:08.300034 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0297bc47-fd57-45b7-b8b6-1498cc6bef65 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:08.300333 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0297bc47-fd57-45b7-b8b6-1498cc6bef65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:08.300652 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0297bc47-fd57-45b7-b8b6-1498cc6bef65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:08.301408 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0297bc47-fd57-45b7-b8b6-1498cc6bef65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:08.303039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0297bc47-fd57-45b7-b8b6-1498cc6bef65 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:08.307623 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:08.322138 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 809/1640] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:08.326867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0297bc47-fd57-45b7-b8b6-1498cc6bef65 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:08.339152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0297bc47-fd57-45b7-b8b6-1498cc6bef65 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:08.353508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0462a-4298-4927-9d89-6b514a37e7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:08.354521 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0462a-4298-4927-9d89-6b514a37e7fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:08.354887 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0462a-4298-4927-9d89-6b514a37e7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:08.355363 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0462a-4298-4927-9d89-6b514a37e7fc admin admin] PATH_INFO: `/v3/projects/2ae43b0e398547308769ff6313f2c12c/users/ae1680bfa99643bcadf69e7b6b64053e/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:08.373455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0462a-4298-4927-9d89-6b514a37e7fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae43b0e398547308769ff6313f2c12c, user_id=ae1680bfa99643bcadf69e7b6b64053e, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:08.378030 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0462a-4298-4927-9d89-6b514a37e7fc admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:08.399498 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d4b1f-d938-421b-b4ad-3ee7926f73ef None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:08.404608 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb703c24-57ae-4317-b37e-c5b40457d8a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:08.405401 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb703c24-57ae-4317-b37e-c5b40457d8a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:08.405992 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb703c24-57ae-4317-b37e-c5b40457d8a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:08.406282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb703c24-57ae-4317-b37e-c5b40457d8a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:08.418705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb703c24-57ae-4317-b37e-c5b40457d8a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:08.420394 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb703c24-57ae-4317-b37e-c5b40457d8a3 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:08.432957 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 832/1641] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:08.456719 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 810/1642] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:08 2022] PUT /identity/v3/projects/2ae43b0e398547308769ff6313f2c12c/users/ae1680bfa99643bcadf69e7b6b64053e/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:45:08.457566 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb703c24-57ae-4317-b37e-c5b40457d8a3 service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:08.469669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb703c24-57ae-4317-b37e-c5b40457d8a3 service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:08.569120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15a5df-f73f-49fe-83da-99555a9bcc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:08.572239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15a5df-f73f-49fe-83da-99555a9bcc1c admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:08.572239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15a5df-f73f-49fe-83da-99555a9bcc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:08.572239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15a5df-f73f-49fe-83da-99555a9bcc1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:08.572239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15a5df-f73f-49fe-83da-99555a9bcc1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:08.573343 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15a5df-f73f-49fe-83da-99555a9bcc1c admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:08.592890 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0462a-4298-4927-9d89-6b514a37e7fc admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:08.593107 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 833/1643] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:45:08.603508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0462a-4298-4927-9d89-6b514a37e7fc admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:08.617322 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbaad14-a503-42a5-aeaf-cc4402523bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:08.618331 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbaad14-a503-42a5-aeaf-cc4402523bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:08.618622 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbaad14-a503-42a5-aeaf-cc4402523bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:08.618901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbaad14-a503-42a5-aeaf-cc4402523bd5 admin admin] PATH_INFO: `/v3/projects/2ae43b0e398547308769ff6313f2c12c/users/ae1680bfa99643bcadf69e7b6b64053e/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:08.621134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbaad14-a503-42a5-aeaf-cc4402523bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae43b0e398547308769ff6313f2c12c, user_id=ae1680bfa99643bcadf69e7b6b64053e, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:08.624469 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbaad14-a503-42a5-aeaf-cc4402523bd5 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:08.642427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 811/1644] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:08 2022] PUT /identity/v3/projects/2ae43b0e398547308769ff6313f2c12c/users/ae1680bfa99643bcadf69e7b6b64053e/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:10.475143 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15a5df-f73f-49fe-83da-99555a9bcc1c admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:10.487657 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15a5df-f73f-49fe-83da-99555a9bcc1c admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:10.652094 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ee235-4969-4421-a1ae-c1eb99c91e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:10.653390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ee235-4969-4421-a1ae-c1eb99c91e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:10.671144 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ee235-4969-4421-a1ae-c1eb99c91e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:10.678639 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ee235-4969-4421-a1ae-c1eb99c91e1c admin admin] PATH_INFO: `/v3/users/a53d7e0b52a84ce5966243f025277da2` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:10.683344 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ee235-4969-4421-a1ae-c1eb99c91e1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a53d7e0b52a84ce5966243f025277da2)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:10.686846 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: WARNING py.warnings [None req-755ee235-4969-4421-a1ae-c1eb99c91e1c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:45:10.686846 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: warnings.warn(msg) Jun 23 22:45:10.686846 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: Jun 23 22:45:10.691014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ee235-4969-4421-a1ae-c1eb99c91e1c admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:11.005240 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:11.042442 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-755ee235-4969-4421-a1ae-c1eb99c91e1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a53d7e0b52a84ce5966243f025277da2'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:11.092167 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-755ee235-4969-4421-a1ae-c1eb99c91e1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a53d7e0b52a84ce5966243f025277da2'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:11.094937 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-755ee235-4969-4421-a1ae-c1eb99c91e1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a53d7e0b52a84ce5966243f025277da2'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:11.096330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-755ee235-4969-4421-a1ae-c1eb99c91e1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a53d7e0b52a84ce5966243f025277da2'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:11.134398 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-755ee235-4969-4421-a1ae-c1eb99c91e1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a53d7e0b52a84ce5966243f025277da2'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:11.271917 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52374d9-d309-445c-bf4e-f1d09d88ca1d None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:11.272437 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52374d9-d309-445c-bf4e-f1d09d88ca1d None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:11.272522 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 834/1645] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:45:10 2022] DELETE /identity/v3/users/a53d7e0b52a84ce5966243f025277da2 => generated 0 bytes in 793 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:11.272645 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52374d9-d309-445c-bf4e-f1d09d88ca1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:11.321480 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-d52374d9-d309-445c-bf4e-f1d09d88ca1d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:11.323418 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-d52374d9-d309-445c-bf4e-f1d09d88ca1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:11.341993 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e914f5b-be11-414d-97db-39512420a688 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:11.342597 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e914f5b-be11-414d-97db-39512420a688 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:11.343036 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e914f5b-be11-414d-97db-39512420a688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:11.406695 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-2e914f5b-be11-414d-97db-39512420a688 None None] MFA Rules not processed for user `ae1680bfa99643bcadf69e7b6b64053e`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:11.407458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-2e914f5b-be11-414d-97db-39512420a688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:11.475832 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 812/1646] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:11.507663 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52374d9-d309-445c-bf4e-f1d09d88ca1d None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:11.509358 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-d52374d9-d309-445c-bf4e-f1d09d88ca1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:11.567779 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 835/1647] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:11 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:45:11.622950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52374d9-d309-445c-bf4e-f1d09d88ca1d None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:11.628234 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5668c-38b6-4159-bc41-ec909e226e3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:11.629228 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5668c-38b6-4159-bc41-ec909e226e3d service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:11.629715 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5668c-38b6-4159-bc41-ec909e226e3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:11.630203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5668c-38b6-4159-bc41-ec909e226e3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:11.643023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5668c-38b6-4159-bc41-ec909e226e3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:11.645295 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5668c-38b6-4159-bc41-ec909e226e3d service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:11.667008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 813/1648] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:13.625596 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8aaaf-f6e8-49ad-8f74-8ac9f7fd9bd8 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:13.629596 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8aaaf-f6e8-49ad-8f74-8ac9f7fd9bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:13.630130 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8aaaf-f6e8-49ad-8f74-8ac9f7fd9bd8 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:13.633666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 836/1649] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 22:45:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:45:13.663952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d20b96-a708-4b1e-9b61-3f784dfb733e None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:13.664867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d20b96-a708-4b1e-9b61-3f784dfb733e None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:13.665363 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d20b96-a708-4b1e-9b61-3f784dfb733e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:13.707451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-61d20b96-a708-4b1e-9b61-3f784dfb733e None None] MFA Rules not processed for user `d3ac44b3f0224e56a15993a2f29c1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:13.709663 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-61d20b96-a708-4b1e-9b61-3f784dfb733e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:13.872596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:13.896317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 814/1650] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 22:45:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:14.037335 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8aaaf-f6e8-49ad-8f74-8ac9f7fd9bd8 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:14.039500 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a2c8aaaf-f6e8-49ad-8f74-8ac9f7fd9bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:14.086041 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8aaaf-f6e8-49ad-8f74-8ac9f7fd9bd8 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:14.131065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312fed0-5914-460d-8cad-c9f34c26913e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ac44b3f0224e56a15993a2f29c1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:14.133019 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312fed0-5914-460d-8cad-c9f34c26913e service placement] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:14.134216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312fed0-5914-460d-8cad-c9f34c26913e service placement] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:14.136910 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312fed0-5914-460d-8cad-c9f34c26913e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:14.164472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1312fed0-5914-460d-8cad-c9f34c26913e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:14.166917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1312fed0-5914-460d-8cad-c9f34c26913e service placement] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:14.206606 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 837/1651] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:14.529638 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d20b96-a708-4b1e-9b61-3f784dfb733e None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:14.531359 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-61d20b96-a708-4b1e-9b61-3f784dfb733e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:14.563753 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d20b96-a708-4b1e-9b61-3f784dfb733e None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:14.595880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d11111-25e6-4dd9-a89d-a66afba2b761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:14.596810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d11111-25e6-4dd9-a89d-a66afba2b761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:14.597098 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d11111-25e6-4dd9-a89d-a66afba2b761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:14.597378 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d11111-25e6-4dd9-a89d-a66afba2b761 admin admin] PATH_INFO: `/v3/projects/ecc700154c034f328172bcdc1487b88c` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:14.602976 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d11111-25e6-4dd9-a89d-a66afba2b761 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecc700154c034f328172bcdc1487b88c)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:14.622777 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: WARNING py.warnings [None req-d9d11111-25e6-4dd9-a89d-a66afba2b761 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:45:14.622777 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: warnings.warn(msg) Jun 23 22:45:14.622777 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: Jun 23 22:45:14.624715 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d11111-25e6-4dd9-a89d-a66afba2b761 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:14.923383 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d9d11111-25e6-4dd9-a89d-a66afba2b761 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecc700154c034f328172bcdc1487b88c', 'request_id': 'req-d9d11111-25e6-4dd9-a89d-a66afba2b761'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:15.044080 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d9d11111-25e6-4dd9-a89d-a66afba2b761 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecc700154c034f328172bcdc1487b88c', 'request_id': 'req-d9d11111-25e6-4dd9-a89d-a66afba2b761'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:15.070668 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d9d11111-25e6-4dd9-a89d-a66afba2b761 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecc700154c034f328172bcdc1487b88c', 'request_id': 'req-d9d11111-25e6-4dd9-a89d-a66afba2b761'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:15.107976 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d9d11111-25e6-4dd9-a89d-a66afba2b761 admin admin] The token cache is being invalidate because project ecc700154c034f328172bcdc1487b88c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:45:15.107976 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d9d11111-25e6-4dd9-a89d-a66afba2b761 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:15.107976 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d9d11111-25e6-4dd9-a89d-a66afba2b761 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:15.107976 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 815/1652] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:45:14 2022] DELETE /identity/v3/projects/ecc700154c034f328172bcdc1487b88c => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:15.108822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfdf97-66e6-4d26-91dd-10e11596d55a None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:15.108822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfdf97-66e6-4d26-91dd-10e11596d55a None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:15.108822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfdf97-66e6-4d26-91dd-10e11596d55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:15.182999 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-01cfdf97-66e6-4d26-91dd-10e11596d55a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:15.185139 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-01cfdf97-66e6-4d26-91dd-10e11596d55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:15.356770 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 838/1653] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:15.360842 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ca26a-0d16-4279-8e14-0a1bfd980a06 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:15.361401 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ca26a-0d16-4279-8e14-0a1bfd980a06 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:15.361928 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ca26a-0d16-4279-8e14-0a1bfd980a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:15.391181 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-134ca26a-0d16-4279-8e14-0a1bfd980a06 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:15.397389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-134ca26a-0d16-4279-8e14-0a1bfd980a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:15.425552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 816/1654] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:15.426233 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cfdf97-66e6-4d26-91dd-10e11596d55a None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:15.440368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cfdf97-66e6-4d26-91dd-10e11596d55a None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:15.467482 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf19d07-9466-4dcb-be1c-7a1efb3a082c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:15.468328 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf19d07-9466-4dcb-be1c-7a1efb3a082c admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:15.468642 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf19d07-9466-4dcb-be1c-7a1efb3a082c admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:15.468930 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf19d07-9466-4dcb-be1c-7a1efb3a082c admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:15.469682 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf19d07-9466-4dcb-be1c-7a1efb3a082c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:45:15.469993 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf19d07-9466-4dcb-be1c-7a1efb3a082c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:15.475028 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf19d07-9466-4dcb-be1c-7a1efb3a082c admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:15.506487 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 839/1655] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:45:15.507857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ca26a-0d16-4279-8e14-0a1bfd980a06 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:15.523490 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ca26a-0d16-4279-8e14-0a1bfd980a06 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:15.534056 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fafa38-0aa3-4fc3-9e4e-812c075d3df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:15.534739 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fafa38-0aa3-4fc3-9e4e-812c075d3df7 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:15.535233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fafa38-0aa3-4fc3-9e4e-812c075d3df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:15.535520 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fafa38-0aa3-4fc3-9e4e-812c075d3df7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:15.536360 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fafa38-0aa3-4fc3-9e4e-812c075d3df7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:15.542345 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fafa38-0aa3-4fc3-9e4e-812c075d3df7 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:15.687463 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 817/1656] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:15 2022] POST /identity/v3/projects => generated 375 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:15.693281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf19d07-9466-4dcb-be1c-7a1efb3a082c admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:15.711330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf19d07-9466-4dcb-be1c-7a1efb3a082c admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:15.729408 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:15.852916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe394d-761e-43e0-a299-3ae1b6fb14bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:15.853862 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe394d-761e-43e0-a299-3ae1b6fb14bc admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:15.854497 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe394d-761e-43e0-a299-3ae1b6fb14bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:15.855044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe394d-761e-43e0-a299-3ae1b6fb14bc admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:15.856737 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe394d-761e-43e0-a299-3ae1b6fb14bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:15.858843 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe394d-761e-43e0-a299-3ae1b6fb14bc admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:15.934078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e30adb-13c8-4f6b-bc57-46451562c53c None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:15.938384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e30adb-13c8-4f6b-bc57-46451562c53c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:15.938384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e30adb-13c8-4f6b-bc57-46451562c53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:15.966062 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 840/1657] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:15 2022] POST /identity/v3/users => generated 288 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:15.969882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe394d-761e-43e0-a299-3ae1b6fb14bc admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:15.981836 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-e7e30adb-13c8-4f6b-bc57-46451562c53c None None] MFA Rules not processed for user `5107e87ee4c34ef9ac96fa4a3a899bda`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:15.982931 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe394d-761e-43e0-a299-3ae1b6fb14bc admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:15.983617 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e7e30adb-13c8-4f6b-bc57-46451562c53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:15.996073 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55432477-9e37-46e6-934e-dee17759f56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:15.996856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55432477-9e37-46e6-934e-dee17759f56e admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:15.997430 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55432477-9e37-46e6-934e-dee17759f56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:15.997430 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55432477-9e37-46e6-934e-dee17759f56e admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:15.998077 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55432477-9e37-46e6-934e-dee17759f56e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:16.000152 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55432477-9e37-46e6-934e-dee17759f56e admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:16.017289 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 841/1658] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:45:16.021226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55432477-9e37-46e6-934e-dee17759f56e admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:16.042561 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55432477-9e37-46e6-934e-dee17759f56e admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:16.042774 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366656cc-cd4b-4d2a-b078-b152f07f042b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:16.043690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366656cc-cd4b-4d2a-b078-b152f07f042b admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:16.043979 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366656cc-cd4b-4d2a-b078-b152f07f042b admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:16.044254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366656cc-cd4b-4d2a-b078-b152f07f042b admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:16.045022 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366656cc-cd4b-4d2a-b078-b152f07f042b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:16.046620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366656cc-cd4b-4d2a-b078-b152f07f042b admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:16.059486 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 842/1659] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:16.065255 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366656cc-cd4b-4d2a-b078-b152f07f042b admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:16.102065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366656cc-cd4b-4d2a-b078-b152f07f042b admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:16.137756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8f315-e5c2-4665-8a0c-417d460fa56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:16.139846 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8f315-e5c2-4665-8a0c-417d460fa56f admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:16.140279 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8f315-e5c2-4665-8a0c-417d460fa56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:16.140752 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8f315-e5c2-4665-8a0c-417d460fa56f admin admin] PATH_INFO: `/v3/projects/34d816bb36c34ae0ad52cfda34b3fa80/users/da5d0201777c49369b1bab15101b3053/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:16.183435 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8f315-e5c2-4665-8a0c-417d460fa56f admin admin] RBAC: Authorizing `identity:create_grant(project_id=34d816bb36c34ae0ad52cfda34b3fa80, user_id=da5d0201777c49369b1bab15101b3053, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:16.192603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8f315-e5c2-4665-8a0c-417d460fa56f admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:16.280469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 843/1660] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:16 2022] PUT /identity/v3/projects/34d816bb36c34ae0ad52cfda34b3fa80/users/da5d0201777c49369b1bab15101b3053/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:16.286593 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 818/1661] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:16.302941 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf4445-5812-482a-97b3-34ff9cea982b None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:16.303441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf4445-5812-482a-97b3-34ff9cea982b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:16.303918 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf4445-5812-482a-97b3-34ff9cea982b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:16.307305 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e30adb-13c8-4f6b-bc57-46451562c53c None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:16.309775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e7e30adb-13c8-4f6b-bc57-46451562c53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:16.377665 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-9ddf4445-5812-482a-97b3-34ff9cea982b None None] MFA Rules not processed for user `da5d0201777c49369b1bab15101b3053`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:16.385298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-9ddf4445-5812-482a-97b3-34ff9cea982b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:16.403390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:16.468993 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e30adb-13c8-4f6b-bc57-46451562c53c None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:16.537767 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 844/1662] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:16.550468 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf4445-5812-482a-97b3-34ff9cea982b None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:16.552166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-9ddf4445-5812-482a-97b3-34ff9cea982b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:16.593123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26997a27-fc86-4271-b922-a977b21b20c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:16.594897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26997a27-fc86-4271-b922-a977b21b20c5 service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:16.595391 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26997a27-fc86-4271-b922-a977b21b20c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:16.595843 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26997a27-fc86-4271-b922-a977b21b20c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:16.666585 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf4445-5812-482a-97b3-34ff9cea982b None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:16.676361 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73af867-7217-459a-94ac-eb0bb915440b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:16.677525 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73af867-7217-459a-94ac-eb0bb915440b service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:16.678247 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73af867-7217-459a-94ac-eb0bb915440b service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:16.678650 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73af867-7217-459a-94ac-eb0bb915440b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:16.690205 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73af867-7217-459a-94ac-eb0bb915440b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:16.692015 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73af867-7217-459a-94ac-eb0bb915440b service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:16.715225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26997a27-fc86-4271-b922-a977b21b20c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:16.716258 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 845/1663] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:16.718322 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26997a27-fc86-4271-b922-a977b21b20c5 service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:16.754125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 819/1664] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:45:16.844239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:17.139685 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73af867-7217-459a-94ac-eb0bb915440b service nova] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:17.216604 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26997a27-fc86-4271-b922-a977b21b20c5 service nova] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:17.219132 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-26997a27-fc86-4271-b922-a977b21b20c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:17.264649 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73af867-7217-459a-94ac-eb0bb915440b service nova] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:17.269993 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a49574-f49f-4a5f-b9e9-97b476ce4df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:17.271307 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a49574-f49f-4a5f-b9e9-97b476ce4df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:17.271816 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a49574-f49f-4a5f-b9e9-97b476ce4df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:17.272312 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a49574-f49f-4a5f-b9e9-97b476ce4df6 admin admin] PATH_INFO: `/v3/users/da5d0201777c49369b1bab15101b3053` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:17.273896 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a49574-f49f-4a5f-b9e9-97b476ce4df6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da5d0201777c49369b1bab15101b3053)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:17.278149 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a49574-f49f-4a5f-b9e9-97b476ce4df6 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:17.420241 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26997a27-fc86-4271-b922-a977b21b20c5 service nova] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:17.427534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5808e5-1906-451e-9ddb-321aa82845f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:17.430781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5808e5-1906-451e-9ddb-321aa82845f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:17.430993 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5808e5-1906-451e-9ddb-321aa82845f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:17.434047 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5808e5-1906-451e-9ddb-321aa82845f4 admin admin] PATH_INFO: `/v3/users/ff743b2ec8db4bc481652a6c8110407b` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:17.436812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5808e5-1906-451e-9ddb-321aa82845f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff743b2ec8db4bc481652a6c8110407b)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:17.442389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5808e5-1906-451e-9ddb-321aa82845f4 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:17.471015 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-96a49574-f49f-4a5f-b9e9-97b476ce4df6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da5d0201777c49369b1bab15101b3053'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:17.503138 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-96a49574-f49f-4a5f-b9e9-97b476ce4df6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da5d0201777c49369b1bab15101b3053'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:17.504143 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-96a49574-f49f-4a5f-b9e9-97b476ce4df6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da5d0201777c49369b1bab15101b3053'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:17.504880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-96a49574-f49f-4a5f-b9e9-97b476ce4df6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da5d0201777c49369b1bab15101b3053'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:17.522303 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-96a49574-f49f-4a5f-b9e9-97b476ce4df6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da5d0201777c49369b1bab15101b3053'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:17.611575 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 846/1665] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:45:17 2022] DELETE /identity/v3/users/da5d0201777c49369b1bab15101b3053 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:17.615987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493de77-0cd7-4655-8936-b818b4bd26e8 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:17.616392 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493de77-0cd7-4655-8936-b818b4bd26e8 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:17.616715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493de77-0cd7-4655-8936-b818b4bd26e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:17.650381 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-d493de77-0cd7-4655-8936-b818b4bd26e8 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:17.654888 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d493de77-0cd7-4655-8936-b818b4bd26e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:17.774560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-df5808e5-1906-451e-9ddb-321aa82845f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff743b2ec8db4bc481652a6c8110407b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:17.804170 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-df5808e5-1906-451e-9ddb-321aa82845f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff743b2ec8db4bc481652a6c8110407b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:17.816466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:17.854973 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 847/1666] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:45:17.867771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493de77-0cd7-4655-8936-b818b4bd26e8 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:17.869705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d493de77-0cd7-4655-8936-b818b4bd26e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:17.900260 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-df5808e5-1906-451e-9ddb-321aa82845f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff743b2ec8db4bc481652a6c8110407b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:17.901152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-df5808e5-1906-451e-9ddb-321aa82845f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff743b2ec8db4bc481652a6c8110407b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:17.902505 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-df5808e5-1906-451e-9ddb-321aa82845f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff743b2ec8db4bc481652a6c8110407b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:17.938516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 820/1667] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:45:17 2022] DELETE /identity/v3/users/ff743b2ec8db4bc481652a6c8110407b => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:17.946303 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:17.992078 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493de77-0cd7-4655-8936-b818b4bd26e8 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:18.122011 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb695609-085c-4072-b7b7-1aff514046cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:18.122838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb695609-085c-4072-b7b7-1aff514046cf service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:18.123228 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb695609-085c-4072-b7b7-1aff514046cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:18.129019 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb695609-085c-4072-b7b7-1aff514046cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:18.133148 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-bb695609-085c-4072-b7b7-1aff514046cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:18.227847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb695609-085c-4072-b7b7-1aff514046cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:18.230388 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb695609-085c-4072-b7b7-1aff514046cf service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:18.252202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 848/1668] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:45:18.431791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5808e5-1906-451e-9ddb-321aa82845f4 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:18.432901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-df5808e5-1906-451e-9ddb-321aa82845f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:18.465128 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5808e5-1906-451e-9ddb-321aa82845f4 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:18.471961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e27eb8-f4a8-4239-9285-454c81a39e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:18.473041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27eb8-f4a8-4239-9285-454c81a39e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:18.473336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27eb8-f4a8-4239-9285-454c81a39e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:18.473634 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27eb8-f4a8-4239-9285-454c81a39e2e admin admin] PATH_INFO: `/v3/projects/9109b68ecbde446ea5fe39f1b4b8df11` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:18.474962 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e27eb8-f4a8-4239-9285-454c81a39e2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9109b68ecbde446ea5fe39f1b4b8df11)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:18.477518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e27eb8-f4a8-4239-9285-454c81a39e2e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:18.619678 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-50e27eb8-f4a8-4239-9285-454c81a39e2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9109b68ecbde446ea5fe39f1b4b8df11', 'request_id': 'req-50e27eb8-f4a8-4239-9285-454c81a39e2e'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:18.735039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-50e27eb8-f4a8-4239-9285-454c81a39e2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9109b68ecbde446ea5fe39f1b4b8df11', 'request_id': 'req-50e27eb8-f4a8-4239-9285-454c81a39e2e'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:18.744851 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-50e27eb8-f4a8-4239-9285-454c81a39e2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9109b68ecbde446ea5fe39f1b4b8df11', 'request_id': 'req-50e27eb8-f4a8-4239-9285-454c81a39e2e'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:18.770244 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-50e27eb8-f4a8-4239-9285-454c81a39e2e admin admin] The token cache is being invalidate because project 9109b68ecbde446ea5fe39f1b4b8df11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:45:18.770606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-50e27eb8-f4a8-4239-9285-454c81a39e2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:18.773924 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-50e27eb8-f4a8-4239-9285-454c81a39e2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:18.781571 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 821/1669] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:45:18 2022] DELETE /identity/v3/projects/9109b68ecbde446ea5fe39f1b4b8df11 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:18.785740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb695609-085c-4072-b7b7-1aff514046cf service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:18.804798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb695609-085c-4072-b7b7-1aff514046cf service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:18.949033 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14448d-4245-4232-aa35-0a5de6bbf371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:18.950234 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14448d-4245-4232-aa35-0a5de6bbf371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:18.950587 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14448d-4245-4232-aa35-0a5de6bbf371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:18.950847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14448d-4245-4232-aa35-0a5de6bbf371 admin admin] PATH_INFO: `/v3/users/ae1680bfa99643bcadf69e7b6b64053e` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:18.953660 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b14448d-4245-4232-aa35-0a5de6bbf371 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae1680bfa99643bcadf69e7b6b64053e)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:18.955684 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b14448d-4245-4232-aa35-0a5de6bbf371 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:19.134572 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e27eb8-f4a8-4239-9285-454c81a39e2e admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:19.136066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-50e27eb8-f4a8-4239-9285-454c81a39e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:19.154199 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e27eb8-f4a8-4239-9285-454c81a39e2e admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:19.160672 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff082a-8f58-4934-b105-0a6c6b57d912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:19.161824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff082a-8f58-4934-b105-0a6c6b57d912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:19.162300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff082a-8f58-4934-b105-0a6c6b57d912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:19.162837 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff082a-8f58-4934-b105-0a6c6b57d912 admin admin] PATH_INFO: `/v3/projects/34d816bb36c34ae0ad52cfda34b3fa80` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:19.164516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff082a-8f58-4934-b105-0a6c6b57d912 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34d816bb36c34ae0ad52cfda34b3fa80)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:19.166990 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff082a-8f58-4934-b105-0a6c6b57d912 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:19.241226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-0b14448d-4245-4232-aa35-0a5de6bbf371 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae1680bfa99643bcadf69e7b6b64053e'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:19.282052 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-0b14448d-4245-4232-aa35-0a5de6bbf371 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae1680bfa99643bcadf69e7b6b64053e'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:19.289746 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d2ff082a-8f58-4934-b105-0a6c6b57d912 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34d816bb36c34ae0ad52cfda34b3fa80', 'request_id': 'req-d2ff082a-8f58-4934-b105-0a6c6b57d912'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:19.299628 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-0b14448d-4245-4232-aa35-0a5de6bbf371 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae1680bfa99643bcadf69e7b6b64053e'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:19.301007 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-0b14448d-4245-4232-aa35-0a5de6bbf371 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae1680bfa99643bcadf69e7b6b64053e'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:19.320182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-0b14448d-4245-4232-aa35-0a5de6bbf371 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae1680bfa99643bcadf69e7b6b64053e'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:19.380481 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d2ff082a-8f58-4934-b105-0a6c6b57d912 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34d816bb36c34ae0ad52cfda34b3fa80', 'request_id': 'req-d2ff082a-8f58-4934-b105-0a6c6b57d912'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:19.389329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d2ff082a-8f58-4934-b105-0a6c6b57d912 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34d816bb36c34ae0ad52cfda34b3fa80', 'request_id': 'req-d2ff082a-8f58-4934-b105-0a6c6b57d912'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:19.403400 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d2ff082a-8f58-4934-b105-0a6c6b57d912 admin admin] The token cache is being invalidate because project 34d816bb36c34ae0ad52cfda34b3fa80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:45:19.403943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d2ff082a-8f58-4934-b105-0a6c6b57d912 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:19.405497 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d2ff082a-8f58-4934-b105-0a6c6b57d912 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:19.411765 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 822/1670] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:45:19 2022] DELETE /identity/v3/projects/34d816bb36c34ae0ad52cfda34b3fa80 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:19.414443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e99131-e70c-4f7c-99ee-04e6854db011 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:19.414901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e99131-e70c-4f7c-99ee-04e6854db011 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:19.418429 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e99131-e70c-4f7c-99ee-04e6854db011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:19.428689 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:DELETE=21,keystone:SELECT=76,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:19.443901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-09e99131-e70c-4f7c-99ee-04e6854db011 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:19.446967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-09e99131-e70c-4f7c-99ee-04e6854db011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:19.506645 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 849/1671] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:45:18 2022] DELETE /identity/v3/users/ae1680bfa99643bcadf69e7b6b64053e => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:45:19.694590 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 823/1672] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:19.713069 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c9e77-5c60-4424-905d-0eb8f029d61e None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:19.713417 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c9e77-5c60-4424-905d-0eb8f029d61e None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:19.713723 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c9e77-5c60-4424-905d-0eb8f029d61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:19.816006 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-238c9e77-5c60-4424-905d-0eb8f029d61e None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:19.821731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-238c9e77-5c60-4424-905d-0eb8f029d61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:19.994174 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:20.004317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 850/1673] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:20.010089 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e99131-e70c-4f7c-99ee-04e6854db011 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:20.022779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e99131-e70c-4f7c-99ee-04e6854db011 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:20.052426 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ccc77f-4d8c-4b5a-809e-e5547ed85983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:20.056897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ccc77f-4d8c-4b5a-809e-e5547ed85983 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:20.057241 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ccc77f-4d8c-4b5a-809e-e5547ed85983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:20.057536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ccc77f-4d8c-4b5a-809e-e5547ed85983 admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:20.058371 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ccc77f-4d8c-4b5a-809e-e5547ed85983 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:45:20.058753 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ccc77f-4d8c-4b5a-809e-e5547ed85983 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:20.060624 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ccc77f-4d8c-4b5a-809e-e5547ed85983 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:20.096937 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 824/1674] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:45:20.098492 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c9e77-5c60-4424-905d-0eb8f029d61e None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:20.114106 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c9e77-5c60-4424-905d-0eb8f029d61e None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:20.133738 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ccc77f-4d8c-4b5a-809e-e5547ed85983 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:20.135740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-b0ccc77f-4d8c-4b5a-809e-e5547ed85983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:20.138013 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d4e83-dbf8-4af7-8881-182b50a959c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:20.138741 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d4e83-dbf8-4af7-8881-182b50a959c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:20.139033 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d4e83-dbf8-4af7-8881-182b50a959c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:20.139320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d4e83-dbf8-4af7-8881-182b50a959c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:20.140042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d4e83-dbf8-4af7-8881-182b50a959c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:20.150120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d4e83-dbf8-4af7-8881-182b50a959c4 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:20.163268 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ccc77f-4d8c-4b5a-809e-e5547ed85983 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:20.178581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8dc2f0-71ba-4a2b-ad7d-53fc9769e697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:20.181722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8dc2f0-71ba-4a2b-ad7d-53fc9769e697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:20.182316 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8dc2f0-71ba-4a2b-ad7d-53fc9769e697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:20.183765 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8dc2f0-71ba-4a2b-ad7d-53fc9769e697 admin admin] PATH_INFO: `/v3/projects/2ae43b0e398547308769ff6313f2c12c` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:20.187979 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8dc2f0-71ba-4a2b-ad7d-53fc9769e697 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ae43b0e398547308769ff6313f2c12c)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:20.193570 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8dc2f0-71ba-4a2b-ad7d-53fc9769e697 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:20.306285 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 851/1675] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:20 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:45:20.307337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d4e83-dbf8-4af7-8881-182b50a959c4 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:20.309174 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-cd1d4e83-dbf8-4af7-8881-182b50a959c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:20.382980 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-4e8dc2f0-71ba-4a2b-ad7d-53fc9769e697 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ae43b0e398547308769ff6313f2c12c', 'request_id': 'req-4e8dc2f0-71ba-4a2b-ad7d-53fc9769e697'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:20.454445 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-4e8dc2f0-71ba-4a2b-ad7d-53fc9769e697 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ae43b0e398547308769ff6313f2c12c', 'request_id': 'req-4e8dc2f0-71ba-4a2b-ad7d-53fc9769e697'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:20.455079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d4e83-dbf8-4af7-8881-182b50a959c4 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:20.466938 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-4e8dc2f0-71ba-4a2b-ad7d-53fc9769e697 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ae43b0e398547308769ff6313f2c12c', 'request_id': 'req-4e8dc2f0-71ba-4a2b-ad7d-53fc9769e697'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:20.491606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-4e8dc2f0-71ba-4a2b-ad7d-53fc9769e697 admin admin] The token cache is being invalidate because project 2ae43b0e398547308769ff6313f2c12c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:45:20.491899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-4e8dc2f0-71ba-4a2b-ad7d-53fc9769e697 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:20.494018 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-4e8dc2f0-71ba-4a2b-ad7d-53fc9769e697 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:20.497995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 825/1676] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:45:20 2022] DELETE /identity/v3/projects/2ae43b0e398547308769ff6313f2c12c => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:45:20.499307 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8dc2f0-71ba-4a2b-ad7d-53fc9769e697 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:20.505166 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-4e8dc2f0-71ba-4a2b-ad7d-53fc9769e697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:20.532243 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:20.578197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a54a96-9ba9-42a9-9921-06ace5ba1d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:20.579782 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a54a96-9ba9-42a9-9921-06ace5ba1d12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:20.580638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a54a96-9ba9-42a9-9921-06ace5ba1d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:20.582526 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a54a96-9ba9-42a9-9921-06ace5ba1d12 admin admin] PATH_INFO: `/v3/users/18861c09a6d241fda06aefcfad0f8eb1` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:20.586869 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a54a96-9ba9-42a9-9921-06ace5ba1d12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18861c09a6d241fda06aefcfad0f8eb1)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:20.591190 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a54a96-9ba9-42a9-9921-06ace5ba1d12 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:20.660512 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8dc2f0-71ba-4a2b-ad7d-53fc9769e697 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:20.666357 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcabc7-7c1b-4141-8e01-a7ac21cfdd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:20.667192 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcabc7-7c1b-4141-8e01-a7ac21cfdd28 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:20.667349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcabc7-7c1b-4141-8e01-a7ac21cfdd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:20.667840 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcabc7-7c1b-4141-8e01-a7ac21cfdd28 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:20.671208 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcabc7-7c1b-4141-8e01-a7ac21cfdd28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:20.673617 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcabc7-7c1b-4141-8e01-a7ac21cfdd28 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:20.787631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 826/1677] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:20 2022] POST /identity/v3/users => generated 294 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:20.787631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680503e-a530-49d3-ba0e-c503dc1ee9a8 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:20.787631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680503e-a530-49d3-ba0e-c503dc1ee9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:20.788030 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680503e-a530-49d3-ba0e-c503dc1ee9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:20.835614 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-4680503e-a530-49d3-ba0e-c503dc1ee9a8 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:20.837983 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-4680503e-a530-49d3-ba0e-c503dc1ee9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:20.838809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b6a54a96-9ba9-42a9-9921-06ace5ba1d12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18861c09a6d241fda06aefcfad0f8eb1'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:20.861324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 827/1678] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:20.862213 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680503e-a530-49d3-ba0e-c503dc1ee9a8 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:20.885971 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b6a54a96-9ba9-42a9-9921-06ace5ba1d12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18861c09a6d241fda06aefcfad0f8eb1'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:20.888760 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b6a54a96-9ba9-42a9-9921-06ace5ba1d12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18861c09a6d241fda06aefcfad0f8eb1'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:20.890948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b6a54a96-9ba9-42a9-9921-06ace5ba1d12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18861c09a6d241fda06aefcfad0f8eb1'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:20.892144 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680503e-a530-49d3-ba0e-c503dc1ee9a8 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:20.903507 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b03dcf-0fa3-426f-9ab1-4489e8bd3131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:20.904213 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b03dcf-0fa3-426f-9ab1-4489e8bd3131 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:20.904476 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b03dcf-0fa3-426f-9ab1-4489e8bd3131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:20.906270 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b03dcf-0fa3-426f-9ab1-4489e8bd3131 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:20.908571 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b6a54a96-9ba9-42a9-9921-06ace5ba1d12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18861c09a6d241fda06aefcfad0f8eb1'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:20.908952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b03dcf-0fa3-426f-9ab1-4489e8bd3131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:20.911978 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b03dcf-0fa3-426f-9ab1-4489e8bd3131 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:20.941821 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 828/1679] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:20.943528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1687645-2a35-4968-bebe-65fc2d3897a8 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:20.943659 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1687645-2a35-4968-bebe-65fc2d3897a8 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:20.943933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1687645-2a35-4968-bebe-65fc2d3897a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:20.991096 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-d1687645-2a35-4968-bebe-65fc2d3897a8 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:20.994835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-d1687645-2a35-4968-bebe-65fc2d3897a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:21.021323 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 829/1680] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:21.022242 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1687645-2a35-4968-bebe-65fc2d3897a8 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:21.023670 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-d1687645-2a35-4968-bebe-65fc2d3897a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:21.029821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 852/1681] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:45:20 2022] DELETE /identity/v3/users/18861c09a6d241fda06aefcfad0f8eb1 => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:21.033432 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a54a96-9ba9-42a9-9921-06ace5ba1d12 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:21.052735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1687645-2a35-4968-bebe-65fc2d3897a8 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:21.153125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec83ae-5f36-490f-9629-6ada3054af1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:21.153823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec83ae-5f36-490f-9629-6ada3054af1e admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:21.154272 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec83ae-5f36-490f-9629-6ada3054af1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:21.154738 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec83ae-5f36-490f-9629-6ada3054af1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:21.155739 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec83ae-5f36-490f-9629-6ada3054af1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:21.162018 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec83ae-5f36-490f-9629-6ada3054af1e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:21.187907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 830/1682] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:21.192100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489a91e-02ad-45f7-bdcb-6d6ec83cfe81 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:21.192629 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489a91e-02ad-45f7-bdcb-6d6ec83cfe81 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:21.195454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489a91e-02ad-45f7-bdcb-6d6ec83cfe81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:21.225989 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-c489a91e-02ad-45f7-bdcb-6d6ec83cfe81 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:21.227497 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c489a91e-02ad-45f7-bdcb-6d6ec83cfe81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:21.242250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 831/1683] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:21.243665 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489a91e-02ad-45f7-bdcb-6d6ec83cfe81 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:21.262496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a54a96-9ba9-42a9-9921-06ace5ba1d12 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:21.277581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489a91e-02ad-45f7-bdcb-6d6ec83cfe81 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:21.277738 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63fbd6-36e3-47c5-a162-16d6f84d0a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:21.278530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63fbd6-36e3-47c5-a162-16d6f84d0a77 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:21.278817 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63fbd6-36e3-47c5-a162-16d6f84d0a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:21.279103 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63fbd6-36e3-47c5-a162-16d6f84d0a77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:21.279852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63fbd6-36e3-47c5-a162-16d6f84d0a77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:45:21.280171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63fbd6-36e3-47c5-a162-16d6f84d0a77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:21.281886 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63fbd6-36e3-47c5-a162-16d6f84d0a77 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:21.288525 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1ac00-5ea1-400a-8a0f-4b2914ede61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:21.290675 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1ac00-5ea1-400a-8a0f-4b2914ede61c admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:21.290675 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1ac00-5ea1-400a-8a0f-4b2914ede61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:21.290675 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1ac00-5ea1-400a-8a0f-4b2914ede61c admin admin] PATH_INFO: `/v3/projects/4df14cc3a2e64414a98009d520e80301/users/3998fafaa5304eb88c194f97cf9e3c7a/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:21.313285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1ac00-5ea1-400a-8a0f-4b2914ede61c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4df14cc3a2e64414a98009d520e80301, user_id=3998fafaa5304eb88c194f97cf9e3c7a, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:21.318787 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1ac00-5ea1-400a-8a0f-4b2914ede61c admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:21.321534 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 853/1684] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:45:21.322628 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63fbd6-36e3-47c5-a162-16d6f84d0a77 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:21.326038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-8b63fbd6-36e3-47c5-a162-16d6f84d0a77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:21.378503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:21.396636 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 832/1685] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:21 2022] PUT /identity/v3/projects/4df14cc3a2e64414a98009d520e80301/users/3998fafaa5304eb88c194f97cf9e3c7a/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:21.396636 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1ac00-5ea1-400a-8a0f-4b2914ede61c admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:21.465555 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63fbd6-36e3-47c5-a162-16d6f84d0a77 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:21.559617 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1ac00-5ea1-400a-8a0f-4b2914ede61c admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:21.578746 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917620f-a69f-4286-be10-e43ec93e890c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:21.578746 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917620f-a69f-4286-be10-e43ec93e890c admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:21.578746 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917620f-a69f-4286-be10-e43ec93e890c admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:21.578746 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917620f-a69f-4286-be10-e43ec93e890c admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:21.588222 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917620f-a69f-4286-be10-e43ec93e890c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:21.596221 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917620f-a69f-4286-be10-e43ec93e890c admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:21.635600 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626be34b-914d-4960-abf3-35310fd59679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:21.642283 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626be34b-914d-4960-abf3-35310fd59679 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:21.642283 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626be34b-914d-4960-abf3-35310fd59679 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:21.642283 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626be34b-914d-4960-abf3-35310fd59679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:21.661961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626be34b-914d-4960-abf3-35310fd59679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:21.663643 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626be34b-914d-4960-abf3-35310fd59679 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:21.699697 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 854/1686] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:21.704914 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93a1de-65ee-4d1d-b4fe-146e3e54bf28 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:21.705712 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93a1de-65ee-4d1d-b4fe-146e3e54bf28 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:21.706132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93a1de-65ee-4d1d-b4fe-146e3e54bf28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:21.746583 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 833/1687] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:21 2022] POST /identity/v3/projects => generated 379 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:21.749542 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917620f-a69f-4286-be10-e43ec93e890c admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:21.757148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-0917620f-a69f-4286-be10-e43ec93e890c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:21.843497 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-ba93a1de-65ee-4d1d-b4fe-146e3e54bf28 None None] MFA Rules not processed for user `3998fafaa5304eb88c194f97cf9e3c7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:21.844339 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ba93a1de-65ee-4d1d-b4fe-146e3e54bf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:22.005195 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917620f-a69f-4286-be10-e43ec93e890c admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:22.018091 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53a47b-a5f6-426a-81c1-5284798be194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:22.021453 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53a47b-a5f6-426a-81c1-5284798be194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:22.024630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53a47b-a5f6-426a-81c1-5284798be194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:22.025137 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53a47b-a5f6-426a-81c1-5284798be194 admin admin] PATH_INFO: `/v3/users/44dd40d10d5846378771bd2307b952fe` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:22.029114 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53a47b-a5f6-426a-81c1-5284798be194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44dd40d10d5846378771bd2307b952fe)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:22.034899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53a47b-a5f6-426a-81c1-5284798be194 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:22.039828 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 855/1688] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:22.042822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93a1de-65ee-4d1d-b4fe-146e3e54bf28 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:22.072047 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93a1de-65ee-4d1d-b4fe-146e3e54bf28 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:22.090728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e360e-4b69-4b1b-b6a4-dc9a9fffeba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:22.091626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e360e-4b69-4b1b-b6a4-dc9a9fffeba7 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:22.091626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e360e-4b69-4b1b-b6a4-dc9a9fffeba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:22.091811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e360e-4b69-4b1b-b6a4-dc9a9fffeba7 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:22.092504 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e360e-4b69-4b1b-b6a4-dc9a9fffeba7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:22.095078 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e360e-4b69-4b1b-b6a4-dc9a9fffeba7 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:22.223388 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 856/1689] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:22 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:22.224301 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e360e-4b69-4b1b-b6a4-dc9a9fffeba7 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:22.251388 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e360e-4b69-4b1b-b6a4-dc9a9fffeba7 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:22.259586 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8be54a-5ece-4e87-8c2f-76c23e087269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:22.260420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8be54a-5ece-4e87-8c2f-76c23e087269 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:22.260736 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8be54a-5ece-4e87-8c2f-76c23e087269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:22.261013 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8be54a-5ece-4e87-8c2f-76c23e087269 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:22.262283 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8be54a-5ece-4e87-8c2f-76c23e087269 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:22.264232 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8be54a-5ece-4e87-8c2f-76c23e087269 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:22.363635 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:22.388395 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9b53a47b-a5f6-426a-81c1-5284798be194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44dd40d10d5846378771bd2307b952fe'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:22.427511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9b53a47b-a5f6-426a-81c1-5284798be194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44dd40d10d5846378771bd2307b952fe'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:22.489216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 857/1690] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:22 2022] POST /identity/v3/projects => generated 387 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:22.489216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8be54a-5ece-4e87-8c2f-76c23e087269 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:22.536224 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9b53a47b-a5f6-426a-81c1-5284798be194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44dd40d10d5846378771bd2307b952fe'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:22.537526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9b53a47b-a5f6-426a-81c1-5284798be194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44dd40d10d5846378771bd2307b952fe'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:22.538230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9b53a47b-a5f6-426a-81c1-5284798be194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44dd40d10d5846378771bd2307b952fe'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:22.575095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 834/1691] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:45:21 2022] DELETE /identity/v3/users/44dd40d10d5846378771bd2307b952fe => generated 0 bytes in 828 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:22.577495 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53a47b-a5f6-426a-81c1-5284798be194 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:22.585346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-9b53a47b-a5f6-426a-81c1-5284798be194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:22.707527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8be54a-5ece-4e87-8c2f-76c23e087269 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:22.781946 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53a47b-a5f6-426a-81c1-5284798be194 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:22.811820 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962a9fd-d9d7-481c-99ab-271ad90779e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:22.812311 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962a9fd-d9d7-481c-99ab-271ad90779e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:22.812732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962a9fd-d9d7-481c-99ab-271ad90779e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:22.813292 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962a9fd-d9d7-481c-99ab-271ad90779e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:22.814009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962a9fd-d9d7-481c-99ab-271ad90779e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:22.820170 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962a9fd-d9d7-481c-99ab-271ad90779e6 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:22.823354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d23f7c-fd85-4b49-8c65-ab60e428f256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:22.824877 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d23f7c-fd85-4b49-8c65-ab60e428f256 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:22.825379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d23f7c-fd85-4b49-8c65-ab60e428f256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:22.825625 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d23f7c-fd85-4b49-8c65-ab60e428f256 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:22.826397 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d23f7c-fd85-4b49-8c65-ab60e428f256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:22.828822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d23f7c-fd85-4b49-8c65-ab60e428f256 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:22.852132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 858/1692] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:22.852132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4420b-2db0-4960-9f13-f7896b6d954c None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:22.852132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4420b-2db0-4960-9f13-f7896b6d954c None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:22.852132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4420b-2db0-4960-9f13-f7896b6d954c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:22.872675 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-81f4420b-2db0-4960-9f13-f7896b6d954c None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:22.876363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-81f4420b-2db0-4960-9f13-f7896b6d954c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:22.893416 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 859/1693] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:22.895863 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4420b-2db0-4960-9f13-f7896b6d954c None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:22.904142 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-81f4420b-2db0-4960-9f13-f7896b6d954c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:22.927383 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4420b-2db0-4960-9f13-f7896b6d954c None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:22.932688 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 835/1694] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:22 2022] POST /identity/v3/users => generated 300 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:22.933751 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962a9fd-d9d7-481c-99ab-271ad90779e6 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:22.937253 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-1962a9fd-d9d7-481c-99ab-271ad90779e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:22.949702 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d98af9-b900-4bdb-94a0-041d6c772215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:22.950824 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d98af9-b900-4bdb-94a0-041d6c772215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:22.951110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d98af9-b900-4bdb-94a0-041d6c772215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:22.951386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d98af9-b900-4bdb-94a0-041d6c772215 admin admin] PATH_INFO: `/v3/projects/fd707e85ae3142e7b2118b13a3587df5` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:22.956756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d98af9-b900-4bdb-94a0-041d6c772215 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd707e85ae3142e7b2118b13a3587df5)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:22.956756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d98af9-b900-4bdb-94a0-041d6c772215 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:22.975571 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962a9fd-d9d7-481c-99ab-271ad90779e6 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:22.990643 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c824d-dd6e-4e6c-8118-0ef147c2b49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:22.991472 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c824d-dd6e-4e6c-8118-0ef147c2b49a admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:22.993095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c824d-dd6e-4e6c-8118-0ef147c2b49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:22.993456 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c824d-dd6e-4e6c-8118-0ef147c2b49a admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:22.996055 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c824d-dd6e-4e6c-8118-0ef147c2b49a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:23.002032 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c824d-dd6e-4e6c-8118-0ef147c2b49a admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:23.036718 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 836/1695] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:23.039973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c824d-dd6e-4e6c-8118-0ef147c2b49a admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:23.071577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c824d-dd6e-4e6c-8118-0ef147c2b49a admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:23.076899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:23.099857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fc6ab-4752-49b2-a441-5a4e14eb3bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:23.100829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fc6ab-4752-49b2-a441-5a4e14eb3bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:23.101877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fc6ab-4752-49b2-a441-5a4e14eb3bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:23.102366 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fc6ab-4752-49b2-a441-5a4e14eb3bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:23.104114 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610fc6ab-4752-49b2-a441-5a4e14eb3bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:23.107511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610fc6ab-4752-49b2-a441-5a4e14eb3bc8 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:23.138135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 837/1696] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:23.142059 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fc6ab-4752-49b2-a441-5a4e14eb3bc8 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:23.149528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-610fc6ab-4752-49b2-a441-5a4e14eb3bc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:23.192622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c2d98af9-b900-4bdb-94a0-041d6c772215 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd707e85ae3142e7b2118b13a3587df5', 'request_id': 'req-c2d98af9-b900-4bdb-94a0-041d6c772215'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:23.232008 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c2d98af9-b900-4bdb-94a0-041d6c772215 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd707e85ae3142e7b2118b13a3587df5', 'request_id': 'req-c2d98af9-b900-4bdb-94a0-041d6c772215'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:23.251651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c2d98af9-b900-4bdb-94a0-041d6c772215 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd707e85ae3142e7b2118b13a3587df5', 'request_id': 'req-c2d98af9-b900-4bdb-94a0-041d6c772215'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:23.338193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c2d98af9-b900-4bdb-94a0-041d6c772215 admin admin] The token cache is being invalidate because project fd707e85ae3142e7b2118b13a3587df5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:45:23.339125 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c2d98af9-b900-4bdb-94a0-041d6c772215 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:23.340878 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c2d98af9-b900-4bdb-94a0-041d6c772215 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:23.347844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 860/1697] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:45:22 2022] DELETE /identity/v3/projects/fd707e85ae3142e7b2118b13a3587df5 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:23.349317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d98af9-b900-4bdb-94a0-041d6c772215 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:23.352476 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-c2d98af9-b900-4bdb-94a0-041d6c772215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:23.396802 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fc6ab-4752-49b2-a441-5a4e14eb3bc8 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:23.516261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d98af9-b900-4bdb-94a0-041d6c772215 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:23.530839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266e79b-d1fc-4d8e-bae6-2d66b144e2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:23.532393 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266e79b-d1fc-4d8e-bae6-2d66b144e2cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:23.532912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266e79b-d1fc-4d8e-bae6-2d66b144e2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:23.533615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266e79b-d1fc-4d8e-bae6-2d66b144e2cd admin admin] PATH_INFO: `/v3/projects/3b618789981848078fb57f45ea89d581/users/944825f6758949288386555fffd7dc07/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:23.552925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02163999-df86-4f52-ad01-2abf2f7f3919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:23.553076 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02163999-df86-4f52-ad01-2abf2f7f3919 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:23.553350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02163999-df86-4f52-ad01-2abf2f7f3919 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:23.553622 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02163999-df86-4f52-ad01-2abf2f7f3919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:23.557020 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-02163999-df86-4f52-ad01-2abf2f7f3919 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:23.575789 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9266e79b-d1fc-4d8e-bae6-2d66b144e2cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b618789981848078fb57f45ea89d581, user_id=944825f6758949288386555fffd7dc07, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:23.579213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9266e79b-d1fc-4d8e-bae6-2d66b144e2cd admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:23.590797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02163999-df86-4f52-ad01-2abf2f7f3919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:23.592248 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02163999-df86-4f52-ad01-2abf2f7f3919 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:23.593291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:23.604000 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 838/1698] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:23.605478 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02163999-df86-4f52-ad01-2abf2f7f3919 service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:23.607318 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-02163999-df86-4f52-ad01-2abf2f7f3919 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:23.626205 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02163999-df86-4f52-ad01-2abf2f7f3919 service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:23.632567 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afd53f-4195-4edb-8f84-d11905fa137f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:23.633523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afd53f-4195-4edb-8f84-d11905fa137f admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:23.633885 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afd53f-4195-4edb-8f84-d11905fa137f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:23.634078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afd53f-4195-4edb-8f84-d11905fa137f admin admin] PATH_INFO: `/v3/projects/2540526ca5f34855b4b2038c5718eab5/users/11b82a77bd534f02b8452af7ee834c96/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:23.661183 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afd53f-4195-4edb-8f84-d11905fa137f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2540526ca5f34855b4b2038c5718eab5, user_id=11b82a77bd534f02b8452af7ee834c96, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:23.663898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afd53f-4195-4edb-8f84-d11905fa137f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:23.687883 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 861/1699] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:23 2022] PUT /identity/v3/projects/3b618789981848078fb57f45ea89d581/users/944825f6758949288386555fffd7dc07/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:23.691711 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cd708-3f58-410b-afac-332e4b04c20b None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:23.692010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cd708-3f58-410b-afac-332e4b04c20b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:23.692340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cd708-3f58-410b-afac-332e4b04c20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:23.715422 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-451cd708-3f58-410b-afac-332e4b04c20b None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:23.716989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-451cd708-3f58-410b-afac-332e4b04c20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:23.738438 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 839/1700] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:23 2022] PUT /identity/v3/projects/2540526ca5f34855b4b2038c5718eab5/users/11b82a77bd534f02b8452af7ee834c96/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:23.740249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b41298-4a69-44d2-a5d9-475a985a38c9 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:23.742396 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b41298-4a69-44d2-a5d9-475a985a38c9 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:23.744616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b41298-4a69-44d2-a5d9-475a985a38c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:23.761978 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:23.815916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-96b41298-4a69-44d2-a5d9-475a985a38c9 None None] MFA Rules not processed for user `944825f6758949288386555fffd7dc07`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:23.819370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-96b41298-4a69-44d2-a5d9-475a985a38c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:23.888513 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 862/1701] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:23.888513 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cd708-3f58-410b-afac-332e4b04c20b None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:23.904432 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cd708-3f58-410b-afac-332e4b04c20b None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:23.928300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 840/1702] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:23.928300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcdd55-79f4-4855-bcae-5c11ab0e455c None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:23.928300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcdd55-79f4-4855-bcae-5c11ab0e455c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:23.928300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcdd55-79f4-4855-bcae-5c11ab0e455c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:23.953214 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-20bcdd55-79f4-4855-bcae-5c11ab0e455c None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:23.954544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-20bcdd55-79f4-4855-bcae-5c11ab0e455c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:24.048234 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8743c1-a2ee-41ca-ae96-9a925750b61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:24.051894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8743c1-a2ee-41ca-ae96-9a925750b61e admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:24.051894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8743c1-a2ee-41ca-ae96-9a925750b61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:24.051894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8743c1-a2ee-41ca-ae96-9a925750b61e admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:24.051894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8743c1-a2ee-41ca-ae96-9a925750b61e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:24.052512 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8743c1-a2ee-41ca-ae96-9a925750b61e admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:24.073135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 841/1703] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:24.074931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcdd55-79f4-4855-bcae-5c11ab0e455c None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:24.094748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 863/1704] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:24.095930 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8743c1-a2ee-41ca-ae96-9a925750b61e admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:24.105385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcdd55-79f4-4855-bcae-5c11ab0e455c None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:24.122373 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de398995-6df4-4a6a-b609-b65925701106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:24.123071 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de398995-6df4-4a6a-b609-b65925701106 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:24.123606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de398995-6df4-4a6a-b609-b65925701106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:24.128090 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de398995-6df4-4a6a-b609-b65925701106 admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:24.129143 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de398995-6df4-4a6a-b609-b65925701106 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:24.129968 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8743c1-a2ee-41ca-ae96-9a925750b61e admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:24.133143 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:24.133354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de398995-6df4-4a6a-b609-b65925701106 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:24.142058 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e8283-6962-42ab-8290-c9fb6f8ee810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:24.142908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e8283-6962-42ab-8290-c9fb6f8ee810 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:24.144209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e8283-6962-42ab-8290-c9fb6f8ee810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:24.145568 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e8283-6962-42ab-8290-c9fb6f8ee810 admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:24.146475 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e8283-6962-42ab-8290-c9fb6f8ee810 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:45:24.148952 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e8283-6962-42ab-8290-c9fb6f8ee810 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:24.154206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e8283-6962-42ab-8290-c9fb6f8ee810 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:24.182702 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 864/1705] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:45:24.183611 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e8283-6962-42ab-8290-c9fb6f8ee810 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:24.194785 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e8283-6962-42ab-8290-c9fb6f8ee810 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:24.201441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e5c66-2dac-454f-a8d7-b0c5c5596255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:24.202359 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e5c66-2dac-454f-a8d7-b0c5c5596255 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:24.202641 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e5c66-2dac-454f-a8d7-b0c5c5596255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:24.202987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e5c66-2dac-454f-a8d7-b0c5c5596255 admin admin] PATH_INFO: `/v3/projects/2540526ca5f34855b4b2038c5718eab5/users/11b82a77bd534f02b8452af7ee834c96/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:24.205173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e5c66-2dac-454f-a8d7-b0c5c5596255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2540526ca5f34855b4b2038c5718eab5, user_id=11b82a77bd534f02b8452af7ee834c96, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:24.208235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e5c66-2dac-454f-a8d7-b0c5c5596255 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:24.232748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 865/1706] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:24 2022] PUT /identity/v3/projects/2540526ca5f34855b4b2038c5718eab5/users/11b82a77bd534f02b8452af7ee834c96/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:24.235719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e5c66-2dac-454f-a8d7-b0c5c5596255 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:24.281110 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 842/1707] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:24 2022] POST /identity/v3/projects => generated 381 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:24.284937 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f3c75-24fd-486f-8f5f-1b2df063d06f None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:24.285246 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f3c75-24fd-486f-8f5f-1b2df063d06f None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:24.286022 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f3c75-24fd-486f-8f5f-1b2df063d06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:24.328683 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-9d6f3c75-24fd-486f-8f5f-1b2df063d06f None None] MFA Rules not processed for user `11b82a77bd534f02b8452af7ee834c96`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:24.340544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-9d6f3c75-24fd-486f-8f5f-1b2df063d06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:24.415967 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e5c66-2dac-454f-a8d7-b0c5c5596255 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:24.524324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b33f5-0a7c-421e-b2ba-906e1536505a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:24.525778 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b33f5-0a7c-421e-b2ba-906e1536505a admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:24.526261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b33f5-0a7c-421e-b2ba-906e1536505a admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:24.527797 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b33f5-0a7c-421e-b2ba-906e1536505a admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:24.528810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b33f5-0a7c-421e-b2ba-906e1536505a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:24.532139 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b33f5-0a7c-421e-b2ba-906e1536505a admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:24.567117 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 843/1708] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:24 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:24.567117 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f3c75-24fd-486f-8f5f-1b2df063d06f None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:24.596541 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f3c75-24fd-486f-8f5f-1b2df063d06f None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:24.603821 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebcdc8-e8ca-4090-904d-26bdec9c6a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:24.604620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebcdc8-e8ca-4090-904d-26bdec9c6a2e admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:24.605216 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebcdc8-e8ca-4090-904d-26bdec9c6a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:24.605513 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebcdc8-e8ca-4090-904d-26bdec9c6a2e admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:24.606466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebcdc8-e8ca-4090-904d-26bdec9c6a2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:24.611206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebcdc8-e8ca-4090-904d-26bdec9c6a2e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:24.643750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 866/1709] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:24 2022] POST /identity/v3/projects => generated 373 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:24.645496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2725ff6-e7ec-49e1-8394-b272a2b7f3d4 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:24.646091 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2725ff6-e7ec-49e1-8394-b272a2b7f3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:24.646531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2725ff6-e7ec-49e1-8394-b272a2b7f3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:24.674427 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-c2725ff6-e7ec-49e1-8394-b272a2b7f3d4 None None] MFA Rules not processed for user `5107e87ee4c34ef9ac96fa4a3a899bda`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:24.675830 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-c2725ff6-e7ec-49e1-8394-b272a2b7f3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:24.703772 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:24.714470 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 844/1710] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:24 2022] POST /identity/v3/users => generated 297 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:24.716060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebcdc8-e8ca-4090-904d-26bdec9c6a2e admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:24.718584 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-22ebcdc8-e8ca-4090-904d-26bdec9c6a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:24.743325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebcdc8-e8ca-4090-904d-26bdec9c6a2e admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:24.798186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 867/1711] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:24.799176 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2725ff6-e7ec-49e1-8394-b272a2b7f3d4 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:24.800884 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-c2725ff6-e7ec-49e1-8394-b272a2b7f3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:24.933934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0f3fbe-7de1-4d41-af51-a69f0393c9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:24.934685 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f3fbe-7de1-4d41-af51-a69f0393c9cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:24.934975 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f3fbe-7de1-4d41-af51-a69f0393c9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:24.935256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f3fbe-7de1-4d41-af51-a69f0393c9cb admin admin] PATH_INFO: `/v3/projects/7bea8c4eba084af1a650862b5fa35276` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:24.938953 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0f3fbe-7de1-4d41-af51-a69f0393c9cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bea8c4eba084af1a650862b5fa35276)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:24.940970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0f3fbe-7de1-4d41-af51-a69f0393c9cb admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:24.964222 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2725ff6-e7ec-49e1-8394-b272a2b7f3d4 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:24.971557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5f54b-5577-4a50-9aac-cfcfcf442998 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:24.973245 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5f54b-5577-4a50-9aac-cfcfcf442998 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:24.973741 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5f54b-5577-4a50-9aac-cfcfcf442998 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:24.974218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5f54b-5577-4a50-9aac-cfcfcf442998 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:25.045100 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:25.126244 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-5c0f3fbe-7de1-4d41-af51-a69f0393c9cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bea8c4eba084af1a650862b5fa35276', 'request_id': 'req-5c0f3fbe-7de1-4d41-af51-a69f0393c9cb'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:25.129550 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf5f54b-5577-4a50-9aac-cfcfcf442998 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:25.132225 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf5f54b-5577-4a50-9aac-cfcfcf442998 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:25.147287 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 868/1712] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:24 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:25.148866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5f54b-5577-4a50-9aac-cfcfcf442998 service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:25.165462 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5f54b-5577-4a50-9aac-cfcfcf442998 service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:25.210173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-5c0f3fbe-7de1-4d41-af51-a69f0393c9cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bea8c4eba084af1a650862b5fa35276', 'request_id': 'req-5c0f3fbe-7de1-4d41-af51-a69f0393c9cb'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:25.219449 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-5c0f3fbe-7de1-4d41-af51-a69f0393c9cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bea8c4eba084af1a650862b5fa35276', 'request_id': 'req-5c0f3fbe-7de1-4d41-af51-a69f0393c9cb'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:25.252922 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-5c0f3fbe-7de1-4d41-af51-a69f0393c9cb admin admin] The token cache is being invalidate because project 7bea8c4eba084af1a650862b5fa35276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:45:25.253400 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-5c0f3fbe-7de1-4d41-af51-a69f0393c9cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:25.256968 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-5c0f3fbe-7de1-4d41-af51-a69f0393c9cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:25.262228 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 845/1713] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:45:24 2022] DELETE /identity/v3/projects/7bea8c4eba084af1a650862b5fa35276 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:25.263327 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0f3fbe-7de1-4d41-af51-a69f0393c9cb admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:25.273905 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-5c0f3fbe-7de1-4d41-af51-a69f0393c9cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:25.300052 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb6d84-56f3-4de0-8820-6c6aa3eb509e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:25.300851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb6d84-56f3-4de0-8820-6c6aa3eb509e admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:25.301695 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb6d84-56f3-4de0-8820-6c6aa3eb509e admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:25.301841 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:25.302296 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb6d84-56f3-4de0-8820-6c6aa3eb509e admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:25.303108 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb6d84-56f3-4de0-8820-6c6aa3eb509e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:25.305683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb6d84-56f3-4de0-8820-6c6aa3eb509e admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:25.388420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 869/1714] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:25 2022] POST /identity/v3/users => generated 287 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:25.388420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c654a05-d6c6-43c2-a873-0138d36a9910 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:25.388420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c654a05-d6c6-43c2-a873-0138d36a9910 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:25.388420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c654a05-d6c6-43c2-a873-0138d36a9910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:25.413870 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0f3fbe-7de1-4d41-af51-a69f0393c9cb admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:25.420838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df9707-80df-4b66-a476-122180f2bcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:25.421710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df9707-80df-4b66-a476-122180f2bcee admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:25.422306 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df9707-80df-4b66-a476-122180f2bcee admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:25.422985 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df9707-80df-4b66-a476-122180f2bcee admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:25.424321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65df9707-80df-4b66-a476-122180f2bcee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:25.426142 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65df9707-80df-4b66-a476-122180f2bcee admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:25.441045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 846/1715] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:25.442304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df9707-80df-4b66-a476-122180f2bcee admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:25.446562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-65df9707-80df-4b66-a476-122180f2bcee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:25.463344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df9707-80df-4b66-a476-122180f2bcee admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:25.463539 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-4c654a05-d6c6-43c2-a873-0138d36a9910 None None] MFA Rules not processed for user `675d117397da452faa5221af0906fb12`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:25.470590 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda844d1-bc8d-45df-9a89-b9a89f5b942b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:25.472008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda844d1-bc8d-45df-9a89-b9a89f5b942b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:25.472443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda844d1-bc8d-45df-9a89-b9a89f5b942b admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:25.472897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda844d1-bc8d-45df-9a89-b9a89f5b942b admin admin] PATH_INFO: `/v3/users/3998fafaa5304eb88c194f97cf9e3c7a` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:25.474485 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda844d1-bc8d-45df-9a89-b9a89f5b942b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3998fafaa5304eb88c194f97cf9e3c7a)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:25.474732 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-4c654a05-d6c6-43c2-a873-0138d36a9910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:25.477170 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda844d1-bc8d-45df-9a89-b9a89f5b942b admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:25.594567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:25.604360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 870/1716] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:25.605961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c654a05-d6c6-43c2-a873-0138d36a9910 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:25.607573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-4c654a05-d6c6-43c2-a873-0138d36a9910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:25.630269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c654a05-d6c6-43c2-a873-0138d36a9910 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:25.635609 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce51d0-37c9-4959-ad89-6aa59e27a799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:25.636737 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce51d0-37c9-4959-ad89-6aa59e27a799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:25.637093 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce51d0-37c9-4959-ad89-6aa59e27a799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:25.637436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce51d0-37c9-4959-ad89-6aa59e27a799 admin admin] PATH_INFO: `/v3/users/1b39eb26fb9644edae3333eba3f9b4f5` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:25.638815 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce51d0-37c9-4959-ad89-6aa59e27a799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b39eb26fb9644edae3333eba3f9b4f5)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:25.640945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce51d0-37c9-4959-ad89-6aa59e27a799 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:25.725564 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-cda844d1-bc8d-45df-9a89-b9a89f5b942b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3998fafaa5304eb88c194f97cf9e3c7a'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:25.762191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-cda844d1-bc8d-45df-9a89-b9a89f5b942b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3998fafaa5304eb88c194f97cf9e3c7a'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:25.893699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-cda844d1-bc8d-45df-9a89-b9a89f5b942b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3998fafaa5304eb88c194f97cf9e3c7a'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:25.903413 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-cda844d1-bc8d-45df-9a89-b9a89f5b942b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3998fafaa5304eb88c194f97cf9e3c7a'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:25.903413 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-cda844d1-bc8d-45df-9a89-b9a89f5b942b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3998fafaa5304eb88c194f97cf9e3c7a'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:25.909780 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d3ce51d0-37c9-4959-ad89-6aa59e27a799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b39eb26fb9644edae3333eba3f9b4f5'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:25.913916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 847/1717] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:45:25 2022] DELETE /identity/v3/users/3998fafaa5304eb88c194f97cf9e3c7a => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:25.915006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda844d1-bc8d-45df-9a89-b9a89f5b942b admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:25.916487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-cda844d1-bc8d-45df-9a89-b9a89f5b942b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:25.935049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d3ce51d0-37c9-4959-ad89-6aa59e27a799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b39eb26fb9644edae3333eba3f9b4f5'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:25.936405 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d3ce51d0-37c9-4959-ad89-6aa59e27a799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b39eb26fb9644edae3333eba3f9b4f5'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:25.937110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d3ce51d0-37c9-4959-ad89-6aa59e27a799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b39eb26fb9644edae3333eba3f9b4f5'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:25.947121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d3ce51d0-37c9-4959-ad89-6aa59e27a799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b39eb26fb9644edae3333eba3f9b4f5'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:25.993691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda844d1-bc8d-45df-9a89-b9a89f5b942b admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:25.999847 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bddff-3c8c-4827-afa7-a2c5ba0b42f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:26.001461 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bddff-3c8c-4827-afa7-a2c5ba0b42f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:26.001461 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bddff-3c8c-4827-afa7-a2c5ba0b42f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:26.001461 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bddff-3c8c-4827-afa7-a2c5ba0b42f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:26.002596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818bddff-3c8c-4827-afa7-a2c5ba0b42f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:26.004091 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818bddff-3c8c-4827-afa7-a2c5ba0b42f6 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:26.015573 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 848/1718] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:26.016488 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bddff-3c8c-4827-afa7-a2c5ba0b42f6 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:26.019841 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-818bddff-3c8c-4827-afa7-a2c5ba0b42f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:26.026284 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 871/1719] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:45:25 2022] DELETE /identity/v3/users/1b39eb26fb9644edae3333eba3f9b4f5 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:26.028195 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823b01b-ef12-4bdb-8204-4617e65fbf7b None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:26.028606 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823b01b-ef12-4bdb-8204-4617e65fbf7b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:26.028994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823b01b-ef12-4bdb-8204-4617e65fbf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:26.037208 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bddff-3c8c-4827-afa7-a2c5ba0b42f6 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:26.047852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-d823b01b-ef12-4bdb-8204-4617e65fbf7b None None] MFA Rules not processed for user `675d117397da452faa5221af0906fb12`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:26.049213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d823b01b-ef12-4bdb-8204-4617e65fbf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:26.107115 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e4a9e-8571-4f05-ac5b-9a6896eae7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:26.108365 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e4a9e-8571-4f05-ac5b-9a6896eae7b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:26.108742 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e4a9e-8571-4f05-ac5b-9a6896eae7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:26.109029 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e4a9e-8571-4f05-ac5b-9a6896eae7b7 admin admin] PATH_INFO: `/v3/projects/13acdb0a10a2448f81e804fc41c1d1af/users/76ead827916f4f048a334f35df362fcf/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:26.128913 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 872/1720] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:26.129188 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e4a9e-8571-4f05-ac5b-9a6896eae7b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13acdb0a10a2448f81e804fc41c1d1af, user_id=76ead827916f4f048a334f35df362fcf, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:26.131311 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcc88f-8020-4941-ba4a-0569a1340419 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:26.131834 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcc88f-8020-4941-ba4a-0569a1340419 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:26.132084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e4a9e-8571-4f05-ac5b-9a6896eae7b7 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:26.132518 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcc88f-8020-4941-ba4a-0569a1340419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:26.138097 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:26.179112 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-21fcc88f-8020-4941-ba4a-0569a1340419 None None] MFA Rules not processed for user `1d7e431402f8433fb7565f69103485ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:26.180391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-21fcc88f-8020-4941-ba4a-0569a1340419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:26.217010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:26.240945 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 849/1721] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:26 2022] PUT /identity/v3/projects/13acdb0a10a2448f81e804fc41c1d1af/users/76ead827916f4f048a334f35df362fcf/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:26.242879 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620603a0-ac77-4189-8268-ae0206891d45 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:26.243382 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620603a0-ac77-4189-8268-ae0206891d45 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:26.244625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620603a0-ac77-4189-8268-ae0206891d45 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:26.246944 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 850/1722] 10.222.0.33 () {54 vars in 912 bytes} [Thu Jun 23 22:45:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:45:26.249296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d03df8-62ab-4608-85e6-b2a5f3c752f5 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:26.249750 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d03df8-62ab-4608-85e6-b2a5f3c752f5 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:26.251400 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d03df8-62ab-4608-85e6-b2a5f3c752f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:26.261512 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 873/1723] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:26.262964 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae39bd0-baf0-4153-841e-3f66c1fc2c55 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:26.263261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae39bd0-baf0-4153-841e-3f66c1fc2c55 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:26.263800 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae39bd0-baf0-4153-841e-3f66c1fc2c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:26.289188 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-c0d03df8-62ab-4608-85e6-b2a5f3c752f5 None None] MFA Rules not processed for user `365b889f8f1a4c5dad09364e5df705ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:26.291332 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c0d03df8-62ab-4608-85e6-b2a5f3c752f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:26.300018 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-7ae39bd0-baf0-4153-841e-3f66c1fc2c55 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:26.301136 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-7ae39bd0-baf0-4153-841e-3f66c1fc2c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:26.414996 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 874/1724] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:26.416642 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae39bd0-baf0-4153-841e-3f66c1fc2c55 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:26.417616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 851/1725] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:26.418605 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-7ae39bd0-baf0-4153-841e-3f66c1fc2c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:26.418998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d03df8-62ab-4608-85e6-b2a5f3c752f5 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:26.425955 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c0d03df8-62ab-4608-85e6-b2a5f3c752f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:26.461082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae39bd0-baf0-4153-841e-3f66c1fc2c55 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:26.485605 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915620f-9b95-48ed-919b-b46c596d6078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:26.486200 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915620f-9b95-48ed-919b-b46c596d6078 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:26.489155 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915620f-9b95-48ed-919b-b46c596d6078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:26.489155 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915620f-9b95-48ed-919b-b46c596d6078 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:26.489155 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2915620f-9b95-48ed-919b-b46c596d6078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:26.489155 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2915620f-9b95-48ed-919b-b46c596d6078 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:26.503986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 875/1726] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:26.505273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915620f-9b95-48ed-919b-b46c596d6078 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:26.524591 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915620f-9b95-48ed-919b-b46c596d6078 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:26.536009 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9769d-48ba-408e-b2cb-adb6d84b773e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:26.536733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9769d-48ba-408e-b2cb-adb6d84b773e admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:26.537023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9769d-48ba-408e-b2cb-adb6d84b773e admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:26.537304 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9769d-48ba-408e-b2cb-adb6d84b773e admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:26.538415 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9769d-48ba-408e-b2cb-adb6d84b773e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:26.541178 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9769d-48ba-408e-b2cb-adb6d84b773e admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:26.554818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 876/1727] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:26.556128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e27eca-9e6d-4939-8914-3d9991735b5f None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:26.556420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e27eca-9e6d-4939-8914-3d9991735b5f None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:26.556776 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e27eca-9e6d-4939-8914-3d9991735b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:26.557114 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d03df8-62ab-4608-85e6-b2a5f3c752f5 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:26.565107 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891b77c-4229-466c-9f39-05799e9446a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:26.565851 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891b77c-4229-466c-9f39-05799e9446a2 service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:26.566222 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891b77c-4229-466c-9f39-05799e9446a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:26.566635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891b77c-4229-466c-9f39-05799e9446a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:26.570695 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-4891b77c-4229-466c-9f39-05799e9446a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:26.627152 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-88e27eca-9e6d-4939-8914-3d9991735b5f None None] MFA Rules not processed for user `ec7412a9a3dc46369fe7b4771e3437de`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:26.629099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-88e27eca-9e6d-4939-8914-3d9991735b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:26.662570 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4891b77c-4229-466c-9f39-05799e9446a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:26.662570 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4891b77c-4229-466c-9f39-05799e9446a2 service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:26.672610 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 852/1728] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:26.685756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891b77c-4229-466c-9f39-05799e9446a2 service nova] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:26.688823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-4891b77c-4229-466c-9f39-05799e9446a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:26.693838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 877/1729] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 22:45:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:26.695758 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106949e7-d531-4b2e-af5e-f1de542eaaac None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:26.696064 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106949e7-d531-4b2e-af5e-f1de542eaaac None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:26.696471 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106949e7-d531-4b2e-af5e-f1de542eaaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:26.708630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891b77c-4229-466c-9f39-05799e9446a2 service nova] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:26.716816 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a1300-5590-488c-afca-7dcb064b2900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:26.718922 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a1300-5590-488c-afca-7dcb064b2900 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:26.719625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a1300-5590-488c-afca-7dcb064b2900 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:26.720626 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a1300-5590-488c-afca-7dcb064b2900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:26.723149 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-106949e7-d531-4b2e-af5e-f1de542eaaac None None] MFA Rules not processed for user `365b889f8f1a4c5dad09364e5df705ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:26.724383 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-106949e7-d531-4b2e-af5e-f1de542eaaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:26.737644 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 878/1730] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:26.737844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a1300-5590-488c-afca-7dcb064b2900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:26.738254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106949e7-d531-4b2e-af5e-f1de542eaaac None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:26.738328 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a1300-5590-488c-afca-7dcb064b2900 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:26.749728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106949e7-d531-4b2e-af5e-f1de542eaaac None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:26.756260 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17dd51-2822-4472-95dd-112f1ca718c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:26.756609 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17dd51-2822-4472-95dd-112f1ca718c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:26.756897 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17dd51-2822-4472-95dd-112f1ca718c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:26.757187 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17dd51-2822-4472-95dd-112f1ca718c1 admin admin] PATH_INFO: `/v3/projects/7ac265d1ab9c42308bbf5441e16597eb/users/a8fe3d4a894b49f3b22515bd77b1ae4c/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:26.759000 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 853/1731] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:26.759981 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a1300-5590-488c-afca-7dcb064b2900 service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:26.770892 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a1300-5590-488c-afca-7dcb064b2900 service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:26.776977 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c85fd-7481-4d77-a935-9195db10da84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:26.779015 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c85fd-7481-4d77-a935-9195db10da84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:26.779015 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c85fd-7481-4d77-a935-9195db10da84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:26.779015 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c85fd-7481-4d77-a935-9195db10da84 admin admin] PATH_INFO: `/v3/projects/13acdb0a10a2448f81e804fc41c1d1af/users/76ead827916f4f048a334f35df362fcf/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:26.782403 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c85fd-7481-4d77-a935-9195db10da84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13acdb0a10a2448f81e804fc41c1d1af, user_id=76ead827916f4f048a334f35df362fcf, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:26.786286 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c85fd-7481-4d77-a935-9195db10da84 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:26.788769 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f17dd51-2822-4472-95dd-112f1ca718c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac265d1ab9c42308bbf5441e16597eb, user_id=a8fe3d4a894b49f3b22515bd77b1ae4c, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:26.791742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f17dd51-2822-4472-95dd-112f1ca718c1 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:26.821574 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 854/1732] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:26 2022] PUT /identity/v3/projects/13acdb0a10a2448f81e804fc41c1d1af/users/76ead827916f4f048a334f35df362fcf/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:26.822721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c85fd-7481-4d77-a935-9195db10da84 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:26.826828 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-185c85fd-7481-4d77-a935-9195db10da84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:26.884522 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 879/1733] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:26 2022] PUT /identity/v3/projects/7ac265d1ab9c42308bbf5441e16597eb/users/a8fe3d4a894b49f3b22515bd77b1ae4c/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:26.896144 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17dd51-2822-4472-95dd-112f1ca718c1 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:26.913640 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c85fd-7481-4d77-a935-9195db10da84 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:26.971840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17dd51-2822-4472-95dd-112f1ca718c1 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:26.978778 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6bb8e-f5ac-4a1c-a8e1-116d72e16f93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:26.980943 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6bb8e-f5ac-4a1c-a8e1-116d72e16f93 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:26.981625 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6bb8e-f5ac-4a1c-a8e1-116d72e16f93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:26.982526 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6bb8e-f5ac-4a1c-a8e1-116d72e16f93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:26.984207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0e3b5-e7ac-4642-aea0-ad45fdada7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:26.985209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0e3b5-e7ac-4642-aea0-ad45fdada7cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:26.985500 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0e3b5-e7ac-4642-aea0-ad45fdada7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:26.986155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0e3b5-e7ac-4642-aea0-ad45fdada7cf admin admin] PATH_INFO: `/v3/projects/5f1871c73f0a44b18154023a2c3e733f` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:26.987136 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b0e3b5-e7ac-4642-aea0-ad45fdada7cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f1871c73f0a44b18154023a2c3e733f)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:26.989667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b0e3b5-e7ac-4642-aea0-ad45fdada7cf admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:27.001846 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6bb8e-f5ac-4a1c-a8e1-116d72e16f93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:27.004472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6bb8e-f5ac-4a1c-a8e1-116d72e16f93 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:27.018358 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 880/1734] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:27.020188 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4715ab5-97a4-4384-9ded-e529068b87c5 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:27.020641 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4715ab5-97a4-4384-9ded-e529068b87c5 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:27.021001 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4715ab5-97a4-4384-9ded-e529068b87c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:27.055169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:27.119975 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-d4715ab5-97a4-4384-9ded-e529068b87c5 None None] MFA Rules not processed for user `76ead827916f4f048a334f35df362fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:27.122824 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d4715ab5-97a4-4384-9ded-e529068b87c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:27.174825 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:27.190177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-15b0e3b5-e7ac-4642-aea0-ad45fdada7cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f1871c73f0a44b18154023a2c3e733f', 'request_id': 'req-15b0e3b5-e7ac-4642-aea0-ad45fdada7cf'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:27.288951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-15b0e3b5-e7ac-4642-aea0-ad45fdada7cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f1871c73f0a44b18154023a2c3e733f', 'request_id': 'req-15b0e3b5-e7ac-4642-aea0-ad45fdada7cf'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:27.296975 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-15b0e3b5-e7ac-4642-aea0-ad45fdada7cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f1871c73f0a44b18154023a2c3e733f', 'request_id': 'req-15b0e3b5-e7ac-4642-aea0-ad45fdada7cf'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:27.317903 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-15b0e3b5-e7ac-4642-aea0-ad45fdada7cf admin admin] The token cache is being invalidate because project 5f1871c73f0a44b18154023a2c3e733f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:45:27.318197 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-15b0e3b5-e7ac-4642-aea0-ad45fdada7cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:27.320353 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-15b0e3b5-e7ac-4642-aea0-ad45fdada7cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:27.320932 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 881/1735] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:27.324160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78927d81-e1d4-4077-b8a2-d692f45f360c None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:27.324537 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78927d81-e1d4-4077-b8a2-d692f45f360c None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:27.324657 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78927d81-e1d4-4077-b8a2-d692f45f360c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:27.336038 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 855/1736] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:45:26 2022] DELETE /identity/v3/projects/5f1871c73f0a44b18154023a2c3e733f => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:27.336038 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0e3b5-e7ac-4642-aea0-ad45fdada7cf admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:27.338344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-15b0e3b5-e7ac-4642-aea0-ad45fdada7cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:27.399418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-78927d81-e1d4-4077-b8a2-d692f45f360c None None] MFA Rules not processed for user `a8fe3d4a894b49f3b22515bd77b1ae4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:27.405528 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-78927d81-e1d4-4077-b8a2-d692f45f360c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:27.472973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0e3b5-e7ac-4642-aea0-ad45fdada7cf admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:27.477322 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fed48-9383-4b61-8927-5623370b55a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:27.478178 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fed48-9383-4b61-8927-5623370b55a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:27.478610 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fed48-9383-4b61-8927-5623370b55a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:27.479034 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fed48-9383-4b61-8927-5623370b55a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:27.481525 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-798fed48-9383-4b61-8927-5623370b55a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:27.546454 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 882/1737] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:27.548381 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9459e-35ad-4049-8f83-43a953afd649 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:27.548729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9459e-35ad-4049-8f83-43a953afd649 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:27.549072 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9459e-35ad-4049-8f83-43a953afd649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:27.575941 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-ecb9459e-35ad-4049-8f83-43a953afd649 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:27.577121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ecb9459e-35ad-4049-8f83-43a953afd649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:27.591195 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798fed48-9383-4b61-8927-5623370b55a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:27.593850 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798fed48-9383-4b61-8927-5623370b55a9 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:27.637427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 856/1738] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:27.641650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb5310-03df-4545-8033-f8c9a4fd9c0b None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:27.643053 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb5310-03df-4545-8033-f8c9a4fd9c0b None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:27.643657 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb5310-03df-4545-8033-f8c9a4fd9c0b None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:27.648001 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 857/1739] 10.222.0.33 () {54 vars in 912 bytes} [Thu Jun 23 22:45:27 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:45:27.649998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb5310-03df-4545-8033-f8c9a4fd9c0b None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:27.654348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-bbcb5310-03df-4545-8033-f8c9a4fd9c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:27.667343 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:27.696460 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 883/1740] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:27.701418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1787a-e2d1-4220-a2ef-04a8857c611a None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:27.701943 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1787a-e2d1-4220-a2ef-04a8857c611a None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:27.702433 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1787a-e2d1-4220-a2ef-04a8857c611a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:27.743220 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-1fd1787a-e2d1-4220-a2ef-04a8857c611a None None] MFA Rules not processed for user `ec7412a9a3dc46369fe7b4771e3437de`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:27.744608 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-1fd1787a-e2d1-4220-a2ef-04a8857c611a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:27.781392 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb5310-03df-4545-8033-f8c9a4fd9c0b None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:27.789569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868fa8e-5682-4855-b1c0-203330e7d545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:27.790253 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868fa8e-5682-4855-b1c0-203330e7d545 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:27.790534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868fa8e-5682-4855-b1c0-203330e7d545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:27.790812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868fa8e-5682-4855-b1c0-203330e7d545 admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:27.791502 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c868fa8e-5682-4855-b1c0-203330e7d545 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:27.793638 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c868fa8e-5682-4855-b1c0-203330e7d545 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:27.836742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 884/1741] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 22:45:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:27.838416 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cab5f-e83b-4306-8551-e32c71f8c046 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:27.838709 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cab5f-e83b-4306-8551-e32c71f8c046 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:27.839479 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cab5f-e83b-4306-8551-e32c71f8c046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:27.862047 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-bc6cab5f-e83b-4306-8551-e32c71f8c046 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:27.863198 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-bc6cab5f-e83b-4306-8551-e32c71f8c046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:27.877539 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 885/1742] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:27.878497 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cab5f-e83b-4306-8551-e32c71f8c046 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:27.881509 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-bc6cab5f-e83b-4306-8551-e32c71f8c046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:27.895640 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cab5f-e83b-4306-8551-e32c71f8c046 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:27.901342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55a13a-2563-45e9-b123-e4a4f40bda6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:27.902785 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55a13a-2563-45e9-b123-e4a4f40bda6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:27.903181 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55a13a-2563-45e9-b123-e4a4f40bda6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:27.903569 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55a13a-2563-45e9-b123-e4a4f40bda6b admin admin] PATH_INFO: `/v3/projects/4df14cc3a2e64414a98009d520e80301` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:27.906397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 858/1743] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:27 2022] POST /identity/v3/projects => generated 371 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:27.906837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55a13a-2563-45e9-b123-e4a4f40bda6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4df14cc3a2e64414a98009d520e80301)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:27.907446 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868fa8e-5682-4855-b1c0-203330e7d545 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:27.909125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c868fa8e-5682-4855-b1c0-203330e7d545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:27.909353 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55a13a-2563-45e9-b123-e4a4f40bda6b admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:28.034974 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868fa8e-5682-4855-b1c0-203330e7d545 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:28.039048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90fbcd-0224-4f53-9bed-d3ffc67afd85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:28.039923 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90fbcd-0224-4f53-9bed-d3ffc67afd85 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:28.040217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90fbcd-0224-4f53-9bed-d3ffc67afd85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:28.040499 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90fbcd-0224-4f53-9bed-d3ffc67afd85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:28.055775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90fbcd-0224-4f53-9bed-d3ffc67afd85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:28.057427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90fbcd-0224-4f53-9bed-d3ffc67afd85 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:28.066981 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 859/1744] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:28.067736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90fbcd-0224-4f53-9bed-d3ffc67afd85 service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:28.101553 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cd55a13a-2563-45e9-b123-e4a4f40bda6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4df14cc3a2e64414a98009d520e80301', 'request_id': 'req-cd55a13a-2563-45e9-b123-e4a4f40bda6b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:28.116192 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cd55a13a-2563-45e9-b123-e4a4f40bda6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4df14cc3a2e64414a98009d520e80301', 'request_id': 'req-cd55a13a-2563-45e9-b123-e4a4f40bda6b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:28.118832 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cd55a13a-2563-45e9-b123-e4a4f40bda6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4df14cc3a2e64414a98009d520e80301', 'request_id': 'req-cd55a13a-2563-45e9-b123-e4a4f40bda6b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:28.136992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90fbcd-0224-4f53-9bed-d3ffc67afd85 service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:28.207942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f52a87-888a-4300-83a0-c1ec474ab1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:28.208845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f52a87-888a-4300-83a0-c1ec474ab1c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:28.209231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f52a87-888a-4300-83a0-c1ec474ab1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:28.209740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f52a87-888a-4300-83a0-c1ec474ab1c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:28.210601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f52a87-888a-4300-83a0-c1ec474ab1c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:45:28.211012 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f52a87-888a-4300-83a0-c1ec474ab1c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:28.212815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f52a87-888a-4300-83a0-c1ec474ab1c4 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:28.226619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 860/1745] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:45:28.227387 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f52a87-888a-4300-83a0-c1ec474ab1c4 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:28.228506 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cd55a13a-2563-45e9-b123-e4a4f40bda6b admin admin] The token cache is being invalidate because project 4df14cc3a2e64414a98009d520e80301 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:45:28.228834 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cd55a13a-2563-45e9-b123-e4a4f40bda6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:28.230515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cd55a13a-2563-45e9-b123-e4a4f40bda6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:28.232969 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 886/1746] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:45:27 2022] DELETE /identity/v3/projects/4df14cc3a2e64414a98009d520e80301 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:28.233920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55a13a-2563-45e9-b123-e4a4f40bda6b admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:28.237027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-cd55a13a-2563-45e9-b123-e4a4f40bda6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:28.257107 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55a13a-2563-45e9-b123-e4a4f40bda6b admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:28.263999 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df252c9-ea75-4e7f-86ea-46aa56c994e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:28.279164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df252c9-ea75-4e7f-86ea-46aa56c994e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:28.280316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df252c9-ea75-4e7f-86ea-46aa56c994e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:28.282321 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df252c9-ea75-4e7f-86ea-46aa56c994e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:28.282321 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df252c9-ea75-4e7f-86ea-46aa56c994e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:28.290487 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df252c9-ea75-4e7f-86ea-46aa56c994e5 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:28.291360 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f52a87-888a-4300-83a0-c1ec474ab1c4 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:28.296756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61113d2-998f-4a3f-a677-97ecf09885d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:28.297584 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61113d2-998f-4a3f-a677-97ecf09885d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:28.298009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61113d2-998f-4a3f-a677-97ecf09885d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:28.298454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61113d2-998f-4a3f-a677-97ecf09885d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:28.300553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c61113d2-998f-4a3f-a677-97ecf09885d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:28.346567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:28.369688 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61113d2-998f-4a3f-a677-97ecf09885d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:28.372785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61113d2-998f-4a3f-a677-97ecf09885d4 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:28.382711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 861/1747] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:28.383639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61113d2-998f-4a3f-a677-97ecf09885d4 service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:28.391198 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c61113d2-998f-4a3f-a677-97ecf09885d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:28.397999 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 887/1748] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:28 2022] POST /identity/v3/users => generated 292 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:28.400426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc1f88-dcb2-45f4-a045-462b68cac288 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:28.400857 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc1f88-dcb2-45f4-a045-462b68cac288 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:28.401504 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc1f88-dcb2-45f4-a045-462b68cac288 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:28.404690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 888/1749] 10.222.0.33 () {54 vars in 960 bytes} [Thu Jun 23 22:45:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:45:28.405578 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc1f88-dcb2-45f4-a045-462b68cac288 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:28.409240 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-22dc1f88-dcb2-45f4-a045-462b68cac288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:28.435317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc1f88-dcb2-45f4-a045-462b68cac288 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:28.440779 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4b1eb-223a-45c2-b10d-fab36679ed35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:28.446917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4b1eb-223a-45c2-b10d-fab36679ed35 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:28.446917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4b1eb-223a-45c2-b10d-fab36679ed35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:28.446917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4b1eb-223a-45c2-b10d-fab36679ed35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:28.446917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4b1eb-223a-45c2-b10d-fab36679ed35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:28.446917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4b1eb-223a-45c2-b10d-fab36679ed35 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:28.532203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61113d2-998f-4a3f-a677-97ecf09885d4 service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:28.537914 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba2d9e-7fe6-4b57-8b28-55cc0ca25bdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:28.538648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba2d9e-7fe6-4b57-8b28-55cc0ca25bdf service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:28.539011 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba2d9e-7fe6-4b57-8b28-55cc0ca25bdf service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:28.539387 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba2d9e-7fe6-4b57-8b28-55cc0ca25bdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:28.541333 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e3ba2d9e-7fe6-4b57-8b28-55cc0ca25bdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:28.585666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 889/1750] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:28 2022] POST /identity/v3/projects => generated 377 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:28.587033 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4b1eb-223a-45c2-b10d-fab36679ed35 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:28.593173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a3d4b1eb-223a-45c2-b10d-fab36679ed35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:28.604066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:28.652999 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba2d9e-7fe6-4b57-8b28-55cc0ca25bdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:28.655838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba2d9e-7fe6-4b57-8b28-55cc0ca25bdf service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:28.669904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 862/1751] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:28.670586 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba2d9e-7fe6-4b57-8b28-55cc0ca25bdf service nova] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:28.703973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba2d9e-7fe6-4b57-8b28-55cc0ca25bdf service nova] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:28.751556 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4b1eb-223a-45c2-b10d-fab36679ed35 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:28.769506 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6b116-ef63-4ab9-8cb0-83a0bd4373bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:28.771112 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6b116-ef63-4ab9-8cb0-83a0bd4373bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:28.771401 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6b116-ef63-4ab9-8cb0-83a0bd4373bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:28.771684 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6b116-ef63-4ab9-8cb0-83a0bd4373bc admin admin] PATH_INFO: `/v3/users/11b82a77bd534f02b8452af7ee834c96` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:28.775416 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6b116-ef63-4ab9-8cb0-83a0bd4373bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=11b82a77bd534f02b8452af7ee834c96)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:28.778290 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6b116-ef63-4ab9-8cb0-83a0bd4373bc admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:28.821240 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243b039-415c-44ce-9fb2-cae7d3dcc819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:28.822021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243b039-415c-44ce-9fb2-cae7d3dcc819 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:28.822372 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243b039-415c-44ce-9fb2-cae7d3dcc819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:28.822717 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243b039-415c-44ce-9fb2-cae7d3dcc819 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:28.823565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243b039-415c-44ce-9fb2-cae7d3dcc819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:28.833987 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243b039-415c-44ce-9fb2-cae7d3dcc819 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:28.845477 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 863/1752] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:28.846777 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a5857-8fa7-4fbd-8a13-2f9f80b8993c None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:28.847222 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a5857-8fa7-4fbd-8a13-2f9f80b8993c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:28.847511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a5857-8fa7-4fbd-8a13-2f9f80b8993c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:28.865885 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-db9a5857-8fa7-4fbd-8a13-2f9f80b8993c None None] MFA Rules not processed for user `d3ac44b3f0224e56a15993a2f29c1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:28.867075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-db9a5857-8fa7-4fbd-8a13-2f9f80b8993c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:28.949404 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 864/1753] 10.222.0.33 () {58 vars in 1057 bytes} [Thu Jun 23 22:45:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:28.950512 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a5857-8fa7-4fbd-8a13-2f9f80b8993c None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:28.969461 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a5857-8fa7-4fbd-8a13-2f9f80b8993c None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:28.974365 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a8f92-5320-4219-ad76-6e20282ae042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:28.975188 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a8f92-5320-4219-ad76-6e20282ae042 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:28.975484 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a8f92-5320-4219-ad76-6e20282ae042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:28.975767 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a8f92-5320-4219-ad76-6e20282ae042 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:28.976470 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a8f92-5320-4219-ad76-6e20282ae042 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:28.978511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a8f92-5320-4219-ad76-6e20282ae042 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:29.022883 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-7ee6b116-ef63-4ab9-8cb0-83a0bd4373bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11b82a77bd534f02b8452af7ee834c96'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:29.058067 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-7ee6b116-ef63-4ab9-8cb0-83a0bd4373bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11b82a77bd534f02b8452af7ee834c96'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:29.061055 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-7ee6b116-ef63-4ab9-8cb0-83a0bd4373bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11b82a77bd534f02b8452af7ee834c96'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:29.062901 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-7ee6b116-ef63-4ab9-8cb0-83a0bd4373bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11b82a77bd534f02b8452af7ee834c96'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:29.078846 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-7ee6b116-ef63-4ab9-8cb0-83a0bd4373bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11b82a77bd534f02b8452af7ee834c96'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:29.083249 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:29.085342 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 865/1754] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:28 2022] POST /identity/v3/users => generated 289 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:29.086529 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a8f92-5320-4219-ad76-6e20282ae042 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:29.090708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-251a8f92-5320-4219-ad76-6e20282ae042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:29.169703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 890/1755] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:45:28 2022] DELETE /identity/v3/users/11b82a77bd534f02b8452af7ee834c96 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:29.170680 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6b116-ef63-4ab9-8cb0-83a0bd4373bc admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:29.172209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-7ee6b116-ef63-4ab9-8cb0-83a0bd4373bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:29.216840 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a8f92-5320-4219-ad76-6e20282ae042 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:29.294789 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb253e-a23a-4892-ab1d-a6692e787eb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:29.295430 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb253e-a23a-4892-ab1d-a6692e787eb4 service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:29.295797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb253e-a23a-4892-ab1d-a6692e787eb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:29.296177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb253e-a23a-4892-ab1d-a6692e787eb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:29.298742 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-65bb253e-a23a-4892-ab1d-a6692e787eb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:29.373832 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb253e-a23a-4892-ab1d-a6692e787eb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:29.375807 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb253e-a23a-4892-ab1d-a6692e787eb4 service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:29.388395 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 866/1756] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:29.389709 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb253e-a23a-4892-ab1d-a6692e787eb4 service nova] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:29.391348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-65bb253e-a23a-4892-ab1d-a6692e787eb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:29.445052 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6b116-ef63-4ab9-8cb0-83a0bd4373bc admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:29.449181 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6284be-66a5-45db-bb57-a7ef4f4bcdd9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ac44b3f0224e56a15993a2f29c1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:29.450426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6284be-66a5-45db-bb57-a7ef4f4bcdd9 service placement] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:29.451548 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6284be-66a5-45db-bb57-a7ef4f4bcdd9 service placement] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:29.452631 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6284be-66a5-45db-bb57-a7ef4f4bcdd9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:29.457382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-5a6284be-66a5-45db-bb57-a7ef4f4bcdd9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:29.472540 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb253e-a23a-4892-ab1d-a6692e787eb4 service nova] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:29.472793 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=82,keystone:DELETE=13,keystone:INSERT=5 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:29.477917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6284be-66a5-45db-bb57-a7ef4f4bcdd9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:29.482111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6284be-66a5-45db-bb57-a7ef4f4bcdd9 service placement] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:29.483357 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12510c98-8b26-496d-8ccf-11e1ac35022a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:29.484284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12510c98-8b26-496d-8ccf-11e1ac35022a admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:29.484571 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12510c98-8b26-496d-8ccf-11e1ac35022a admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:29.484890 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12510c98-8b26-496d-8ccf-11e1ac35022a admin admin] PATH_INFO: `/v3/projects/5a8ff6a49e69456f9a6c28bb157c79fc/users/8c5d929ec1004638ae06b8aaf6cd55ef/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:29.510487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12510c98-8b26-496d-8ccf-11e1ac35022a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a8ff6a49e69456f9a6c28bb157c79fc, user_id=8c5d929ec1004638ae06b8aaf6cd55ef, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:29.513337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 891/1757] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:29.514177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12510c98-8b26-496d-8ccf-11e1ac35022a admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:29.514641 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6284be-66a5-45db-bb57-a7ef4f4bcdd9 service placement] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:29.517866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-5a6284be-66a5-45db-bb57-a7ef4f4bcdd9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:29.548115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6284be-66a5-45db-bb57-a7ef4f4bcdd9 service placement] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:29.560473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54715369-9825-4591-8764-cb7dc9bbd2f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ac44b3f0224e56a15993a2f29c1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:29.561172 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54715369-9825-4591-8764-cb7dc9bbd2f8 service placement] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:29.561461 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54715369-9825-4591-8764-cb7dc9bbd2f8 service placement] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:29.561738 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54715369-9825-4591-8764-cb7dc9bbd2f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:29.585011 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54715369-9825-4591-8764-cb7dc9bbd2f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:29.586897 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54715369-9825-4591-8764-cb7dc9bbd2f8 service placement] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:29.599821 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 867/1758] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:29 2022] PUT /identity/v3/projects/5a8ff6a49e69456f9a6c28bb157c79fc/users/8c5d929ec1004638ae06b8aaf6cd55ef/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:29.601232 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12510c98-8b26-496d-8ccf-11e1ac35022a admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:29.603982 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-12510c98-8b26-496d-8ccf-11e1ac35022a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:29.618375 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 892/1759] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:29.619862 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc51dc7-0277-4a2c-b4f5-5f6fc687d71c None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:29.620183 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc51dc7-0277-4a2c-b4f5-5f6fc687d71c None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:29.620458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc51dc7-0277-4a2c-b4f5-5f6fc687d71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:29.659968 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-fdc51dc7-0277-4a2c-b4f5-5f6fc687d71c None None] MFA Rules not processed for user `675d117397da452faa5221af0906fb12`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:29.664481 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-fdc51dc7-0277-4a2c-b4f5-5f6fc687d71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:29.729298 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:29.782224 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12510c98-8b26-496d-8ccf-11e1ac35022a admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:29.796011 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8b56f-0e02-49c3-ba89-2383cf9e4c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:29.796762 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8b56f-0e02-49c3-ba89-2383cf9e4c28 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:29.797704 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8b56f-0e02-49c3-ba89-2383cf9e4c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:29.798299 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8b56f-0e02-49c3-ba89-2383cf9e4c28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:29.799190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8b56f-0e02-49c3-ba89-2383cf9e4c28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:29.804411 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8b56f-0e02-49c3-ba89-2383cf9e4c28 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:29.809057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 893/1760] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:29.825356 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc51dc7-0277-4a2c-b4f5-5f6fc687d71c None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:29.850773 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 868/1761] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:29.852102 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8b56f-0e02-49c3-ba89-2383cf9e4c28 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:29.863275 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc51dc7-0277-4a2c-b4f5-5f6fc687d71c None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:29.873661 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3774b-1e33-47cd-b48c-ff963cb467d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:29.874350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3774b-1e33-47cd-b48c-ff963cb467d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:29.874631 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3774b-1e33-47cd-b48c-ff963cb467d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:29.874913 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3774b-1e33-47cd-b48c-ff963cb467d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:29.875653 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3774b-1e33-47cd-b48c-ff963cb467d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:29.889788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3774b-1e33-47cd-b48c-ff963cb467d2 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:29.900996 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8b56f-0e02-49c3-ba89-2383cf9e4c28 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:29.922436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e95eab-9d52-4a77-88be-6f851b31c1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:29.925777 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e95eab-9d52-4a77-88be-6f851b31c1b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:29.926467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e95eab-9d52-4a77-88be-6f851b31c1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:29.926706 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e95eab-9d52-4a77-88be-6f851b31c1b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:29.929988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e95eab-9d52-4a77-88be-6f851b31c1b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:29.932254 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e95eab-9d52-4a77-88be-6f851b31c1b7 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:29.933903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 894/1762] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:29.937009 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3774b-1e33-47cd-b48c-ff963cb467d2 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:29.968023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3774b-1e33-47cd-b48c-ff963cb467d2 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:29.974387 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 869/1763] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:29.977325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e95eab-9d52-4a77-88be-6f851b31c1b7 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:30.009639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e95eab-9d52-4a77-88be-6f851b31c1b7 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:30.013510 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe795b9-7efa-4509-bbc0-b98f141ee7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:30.020762 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe795b9-7efa-4509-bbc0-b98f141ee7ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:30.020988 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe795b9-7efa-4509-bbc0-b98f141ee7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:30.021267 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe795b9-7efa-4509-bbc0-b98f141ee7ac admin admin] PATH_INFO: `/v3/projects/5a8ff6a49e69456f9a6c28bb157c79fc/users/8c5d929ec1004638ae06b8aaf6cd55ef/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:30.028205 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe795b9-7efa-4509-bbc0-b98f141ee7ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a8ff6a49e69456f9a6c28bb157c79fc, user_id=8c5d929ec1004638ae06b8aaf6cd55ef, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:30.034102 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6848343-29b5-478d-b6ff-fa010f4bd76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:30.035028 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6848343-29b5-478d-b6ff-fa010f4bd76a admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:30.035366 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6848343-29b5-478d-b6ff-fa010f4bd76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:30.035625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6848343-29b5-478d-b6ff-fa010f4bd76a admin admin] PATH_INFO: `/v3/projects/3711628a625146f388cb276e025da7de/users/43fd115fc9d5481eb3228327360c111f/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:30.051336 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe795b9-7efa-4509-bbc0-b98f141ee7ac admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:30.103078 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 895/1764] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:29 2022] PUT /identity/v3/projects/5a8ff6a49e69456f9a6c28bb157c79fc/users/8c5d929ec1004638ae06b8aaf6cd55ef/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:30.109681 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6848343-29b5-478d-b6ff-fa010f4bd76a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3711628a625146f388cb276e025da7de, user_id=43fd115fc9d5481eb3228327360c111f, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:30.113519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc2177-90d3-4d15-8277-a6e37fb37025 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:30.113808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc2177-90d3-4d15-8277-a6e37fb37025 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:30.115960 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc2177-90d3-4d15-8277-a6e37fb37025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:30.118146 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6848343-29b5-478d-b6ff-fa010f4bd76a admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:30.206959 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-e6cc2177-90d3-4d15-8277-a6e37fb37025 None None] MFA Rules not processed for user `8c5d929ec1004638ae06b8aaf6cd55ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:30.208135 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e6cc2177-90d3-4d15-8277-a6e37fb37025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:30.248800 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 870/1765] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:29 2022] PUT /identity/v3/projects/3711628a625146f388cb276e025da7de/users/43fd115fc9d5481eb3228327360c111f/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:45:30.249413 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:30.253699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6848343-29b5-478d-b6ff-fa010f4bd76a admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:30.257641 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e6848343-29b5-478d-b6ff-fa010f4bd76a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:30.331180 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:30.378114 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 896/1766] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:30 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:30.381329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586f83e-5798-404f-a4fa-6a35dd0223b6 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:30.381626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586f83e-5798-404f-a4fa-6a35dd0223b6 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:30.383352 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586f83e-5798-404f-a4fa-6a35dd0223b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:30.398561 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6848343-29b5-478d-b6ff-fa010f4bd76a admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:30.404972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342ac38-0013-405a-b98f-0c1ac40217e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:30.406125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342ac38-0013-405a-b98f-0c1ac40217e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:30.406416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342ac38-0013-405a-b98f-0c1ac40217e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:30.407176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342ac38-0013-405a-b98f-0c1ac40217e2 admin admin] PATH_INFO: `/v3/projects/2540526ca5f34855b4b2038c5718eab5` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:30.408172 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342ac38-0013-405a-b98f-0c1ac40217e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2540526ca5f34855b4b2038c5718eab5)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:30.410427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342ac38-0013-405a-b98f-0c1ac40217e2 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:30.484160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-f586f83e-5798-404f-a4fa-6a35dd0223b6 None None] MFA Rules not processed for user `43fd115fc9d5481eb3228327360c111f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:30.485705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-f586f83e-5798-404f-a4fa-6a35dd0223b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:30.587316 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7342ac38-0013-405a-b98f-0c1ac40217e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2540526ca5f34855b4b2038c5718eab5', 'request_id': 'req-7342ac38-0013-405a-b98f-0c1ac40217e2'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:30.673427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:30.677049 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7342ac38-0013-405a-b98f-0c1ac40217e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2540526ca5f34855b4b2038c5718eab5', 'request_id': 'req-7342ac38-0013-405a-b98f-0c1ac40217e2'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:30.693546 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7342ac38-0013-405a-b98f-0c1ac40217e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2540526ca5f34855b4b2038c5718eab5', 'request_id': 'req-7342ac38-0013-405a-b98f-0c1ac40217e2'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:30.773876 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7342ac38-0013-405a-b98f-0c1ac40217e2 admin admin] The token cache is being invalidate because project 2540526ca5f34855b4b2038c5718eab5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:45:30.777377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7342ac38-0013-405a-b98f-0c1ac40217e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:30.781218 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7342ac38-0013-405a-b98f-0c1ac40217e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:30.796457 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 871/1767] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:45:30 2022] DELETE /identity/v3/projects/2540526ca5f34855b4b2038c5718eab5 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:30.799444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342ac38-0013-405a-b98f-0c1ac40217e2 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:30.806220 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-7342ac38-0013-405a-b98f-0c1ac40217e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:30.851184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 897/1768] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:30.853716 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ff6e0-fb0c-4ded-9d5d-625ea5da3653 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:30.854376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ff6e0-fb0c-4ded-9d5d-625ea5da3653 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:30.858746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ff6e0-fb0c-4ded-9d5d-625ea5da3653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:30.929618 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-a55ff6e0-fb0c-4ded-9d5d-625ea5da3653 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:30.931100 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a55ff6e0-fb0c-4ded-9d5d-625ea5da3653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:31.058012 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342ac38-0013-405a-b98f-0c1ac40217e2 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:31.097055 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef57af0-4959-4968-a4c2-4c0fcc501da2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:31.106780 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 898/1769] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:31.106780 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ff6e0-fb0c-4ded-9d5d-625ea5da3653 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:31.107095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef57af0-4959-4968-a4c2-4c0fcc501da2 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:31.107095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef57af0-4959-4968-a4c2-4c0fcc501da2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:31.107095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef57af0-4959-4968-a4c2-4c0fcc501da2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:31.110394 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a55ff6e0-fb0c-4ded-9d5d-625ea5da3653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:31.115429 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-aef57af0-4959-4968-a4c2-4c0fcc501da2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:31.147372 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ff6e0-fb0c-4ded-9d5d-625ea5da3653 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:31.165948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f096c3-9247-4d5a-8583-465f87b0deb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:31.166923 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f096c3-9247-4d5a-8583-465f87b0deb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:31.167546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f096c3-9247-4d5a-8583-465f87b0deb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:31.167999 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f096c3-9247-4d5a-8583-465f87b0deb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:31.168932 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f096c3-9247-4d5a-8583-465f87b0deb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:31.171050 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f096c3-9247-4d5a-8583-465f87b0deb9 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:31.304082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 899/1770] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:31 2022] POST /identity/v3/projects => generated 377 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:31.315086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f20cfc-c1d9-42f2-ba0b-8e28d0226aeb None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:31.315402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f20cfc-c1d9-42f2-ba0b-8e28d0226aeb None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:31.315690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f20cfc-c1d9-42f2-ba0b-8e28d0226aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:31.421338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-47f20cfc-c1d9-42f2-ba0b-8e28d0226aeb None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:31.422577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-47f20cfc-c1d9-42f2-ba0b-8e28d0226aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:31.434742 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef57af0-4959-4968-a4c2-4c0fcc501da2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:31.440882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef57af0-4959-4968-a4c2-4c0fcc501da2 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:31.521725 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 872/1771] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:30 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 721 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:31.521725 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef57af0-4959-4968-a4c2-4c0fcc501da2 service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:31.574324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef57af0-4959-4968-a4c2-4c0fcc501da2 service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:31.622675 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:31.712119 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:31.718349 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 900/1772] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:31.729838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f20cfc-c1d9-42f2-ba0b-8e28d0226aeb None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:31.735592 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05496b65-1677-4214-a41d-bc2e5ba68005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:31.736297 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05496b65-1677-4214-a41d-bc2e5ba68005 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:31.736645 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05496b65-1677-4214-a41d-bc2e5ba68005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:31.736880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05496b65-1677-4214-a41d-bc2e5ba68005 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:31.737583 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05496b65-1677-4214-a41d-bc2e5ba68005 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:31.745995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05496b65-1677-4214-a41d-bc2e5ba68005 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:31.765890 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f20cfc-c1d9-42f2-ba0b-8e28d0226aeb None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:31.788437 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce36bdf-248e-4cec-b081-69f4dd78b4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:31.789077 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce36bdf-248e-4cec-b081-69f4dd78b4f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:31.789400 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce36bdf-248e-4cec-b081-69f4dd78b4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:31.789684 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce36bdf-248e-4cec-b081-69f4dd78b4f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:31.793821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce36bdf-248e-4cec-b081-69f4dd78b4f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:45:31.799110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce36bdf-248e-4cec-b081-69f4dd78b4f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:31.799110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce36bdf-248e-4cec-b081-69f4dd78b4f4 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:31.838431 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 901/1773] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:45:31.839922 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce36bdf-248e-4cec-b081-69f4dd78b4f4 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:31.862603 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 873/1774] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:31 2022] POST /identity/v3/users => generated 295 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:31.867248 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05496b65-1677-4214-a41d-bc2e5ba68005 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:31.881312 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce36bdf-248e-4cec-b081-69f4dd78b4f4 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:31.905039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05496b65-1677-4214-a41d-bc2e5ba68005 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:31.908609 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e7436-2f36-4922-940d-71cdb5d1aec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:31.909345 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e7436-2f36-4922-940d-71cdb5d1aec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:31.909903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e7436-2f36-4922-940d-71cdb5d1aec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:31.911326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e7436-2f36-4922-940d-71cdb5d1aec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:31.912052 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e7436-2f36-4922-940d-71cdb5d1aec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:31.914954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e7436-2f36-4922-940d-71cdb5d1aec7 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:31.922051 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729482d3-ff98-4811-989c-71d0447bab39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:31.922879 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729482d3-ff98-4811-989c-71d0447bab39 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:31.925121 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729482d3-ff98-4811-989c-71d0447bab39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:31.925121 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729482d3-ff98-4811-989c-71d0447bab39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:31.925121 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729482d3-ff98-4811-989c-71d0447bab39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:31.931530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729482d3-ff98-4811-989c-71d0447bab39 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:31.984550 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 874/1775] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:31.988413 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729482d3-ff98-4811-989c-71d0447bab39 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:32.069182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 902/1776] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:31 2022] POST /identity/v3/projects => generated 375 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:32.070131 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e7436-2f36-4922-940d-71cdb5d1aec7 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:32.156849 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729482d3-ff98-4811-989c-71d0447bab39 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:32.234157 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e7436-2f36-4922-940d-71cdb5d1aec7 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:32.247517 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29bf77-265f-43d5-872a-15bfec6126dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:32.250387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29bf77-265f-43d5-872a-15bfec6126dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:32.250685 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29bf77-265f-43d5-872a-15bfec6126dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:32.250986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29bf77-265f-43d5-872a-15bfec6126dc admin admin] PATH_INFO: `/v3/projects/7f2eb7f2ca5c45c3bb9aedcdf6362fff/users/80ed460262494822a370a5b70b5b9383/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:32.310753 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29bf77-265f-43d5-872a-15bfec6126dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f2eb7f2ca5c45c3bb9aedcdf6362fff, user_id=80ed460262494822a370a5b70b5b9383, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:32.314391 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29bf77-265f-43d5-872a-15bfec6126dc admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:32.322789 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e5bcc-c122-49c3-a3e1-929c2b1de90d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:32.324102 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e5bcc-c122-49c3-a3e1-929c2b1de90d service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:32.324274 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e5bcc-c122-49c3-a3e1-929c2b1de90d service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:32.324557 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e5bcc-c122-49c3-a3e1-929c2b1de90d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:32.327489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-281e5bcc-c122-49c3-a3e1-929c2b1de90d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:32.411005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 903/1777] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:32 2022] PUT /identity/v3/projects/7f2eb7f2ca5c45c3bb9aedcdf6362fff/users/80ed460262494822a370a5b70b5b9383/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:32.412053 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29bf77-265f-43d5-872a-15bfec6126dc admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:32.469875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e5bcc-c122-49c3-a3e1-929c2b1de90d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:32.471878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e5bcc-c122-49c3-a3e1-929c2b1de90d service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:32.529033 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 875/1778] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:32.532395 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e5bcc-c122-49c3-a3e1-929c2b1de90d service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:32.583593 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e5bcc-c122-49c3-a3e1-929c2b1de90d service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:32.595262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29bf77-265f-43d5-872a-15bfec6126dc admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:32.603795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef63d1-be89-4706-92c4-5cfd9fac6fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:32.606625 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef63d1-be89-4706-92c4-5cfd9fac6fcd admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:32.607155 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef63d1-be89-4706-92c4-5cfd9fac6fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:32.607806 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef63d1-be89-4706-92c4-5cfd9fac6fcd admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:32.609573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef63d1-be89-4706-92c4-5cfd9fac6fcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:32.613651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef63d1-be89-4706-92c4-5cfd9fac6fcd admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:32.614518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad2e29-c132-42a8-97a4-e05b87d33228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:32.614653 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad2e29-c132-42a8-97a4-e05b87d33228 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:32.614952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad2e29-c132-42a8-97a4-e05b87d33228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:32.615551 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad2e29-c132-42a8-97a4-e05b87d33228 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:32.616015 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad2e29-c132-42a8-97a4-e05b87d33228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:32.627919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad2e29-c132-42a8-97a4-e05b87d33228 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:32.671367 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 876/1779] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:32.681652 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad2e29-c132-42a8-97a4-e05b87d33228 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:32.711472 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad2e29-c132-42a8-97a4-e05b87d33228 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:32.736914 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21ca93-bd46-4f96-90a3-739ca5a7cc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:32.737907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21ca93-bd46-4f96-90a3-739ca5a7cc7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:32.738198 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21ca93-bd46-4f96-90a3-739ca5a7cc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:32.738485 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21ca93-bd46-4f96-90a3-739ca5a7cc7f admin admin] PATH_INFO: `/v3/projects/7f2eb7f2ca5c45c3bb9aedcdf6362fff/users/80ed460262494822a370a5b70b5b9383/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:32.746438 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21ca93-bd46-4f96-90a3-739ca5a7cc7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f2eb7f2ca5c45c3bb9aedcdf6362fff, user_id=80ed460262494822a370a5b70b5b9383, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:32.749544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21ca93-bd46-4f96-90a3-739ca5a7cc7f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:32.773209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 904/1780] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:32 2022] POST /identity/v3/users => generated 288 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:32.777447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef63d1-be89-4706-92c4-5cfd9fac6fcd admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:32.784198 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 877/1781] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:32 2022] PUT /identity/v3/projects/7f2eb7f2ca5c45c3bb9aedcdf6362fff/users/80ed460262494822a370a5b70b5b9383/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:32.798092 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ba92d-0e13-4d4b-affb-91b641915c45 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:32.798619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ba92d-0e13-4d4b-affb-91b641915c45 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:32.799067 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ba92d-0e13-4d4b-affb-91b641915c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:32.805235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef63d1-be89-4706-92c4-5cfd9fac6fcd admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:32.877818 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-976ba92d-0e13-4d4b-affb-91b641915c45 None None] MFA Rules not processed for user `80ed460262494822a370a5b70b5b9383`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:32.879180 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-976ba92d-0e13-4d4b-affb-91b641915c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:32.900109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:32.986880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445821eb-bc57-4c88-928c-62b8a8a6850f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:33.005513 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445821eb-bc57-4c88-928c-62b8a8a6850f admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:33.009869 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445821eb-bc57-4c88-928c-62b8a8a6850f admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:33.011483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445821eb-bc57-4c88-928c-62b8a8a6850f admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:33.012301 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445821eb-bc57-4c88-928c-62b8a8a6850f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:33.026087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445821eb-bc57-4c88-928c-62b8a8a6850f admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:33.043383 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:33.105961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 878/1782] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:33.116489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ba92d-0e13-4d4b-affb-91b641915c45 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:33.119064 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 905/1783] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:33.155932 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445821eb-bc57-4c88-928c-62b8a8a6850f admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:33.156999 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ba92d-0e13-4d4b-affb-91b641915c45 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:33.174691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89226b-c289-4e4c-a7b8-b9d7d28ed11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:33.175619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89226b-c289-4e4c-a7b8-b9d7d28ed11b admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:33.176169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89226b-c289-4e4c-a7b8-b9d7d28ed11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:33.176667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89226b-c289-4e4c-a7b8-b9d7d28ed11b admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:33.177718 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89226b-c289-4e4c-a7b8-b9d7d28ed11b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:33.180054 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89226b-c289-4e4c-a7b8-b9d7d28ed11b admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:33.204192 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 879/1784] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:33.217244 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89226b-c289-4e4c-a7b8-b9d7d28ed11b admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:33.256100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89226b-c289-4e4c-a7b8-b9d7d28ed11b admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:33.266982 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1e08f-1d1f-4ed5-9a0c-a8c4b28f237e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:33.268044 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1e08f-1d1f-4ed5-9a0c-a8c4b28f237e admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:33.268213 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1e08f-1d1f-4ed5-9a0c-a8c4b28f237e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:33.268499 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1e08f-1d1f-4ed5-9a0c-a8c4b28f237e admin admin] PATH_INFO: `/v3/projects/2afc430f1d89469bb55935ca6f1abfe6/users/064efa62b57c4a0c92ed9480aa0d6988/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:33.305737 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445821eb-bc57-4c88-928c-62b8a8a6850f admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:33.311730 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1e08f-1d1f-4ed5-9a0c-a8c4b28f237e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2afc430f1d89469bb55935ca6f1abfe6, user_id=064efa62b57c4a0c92ed9480aa0d6988, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:33.316280 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e769d8-1804-42c9-b51f-17b97a6b7ecc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:33.317943 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1e08f-1d1f-4ed5-9a0c-a8c4b28f237e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:33.318405 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e769d8-1804-42c9-b51f-17b97a6b7ecc service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:33.319017 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e769d8-1804-42c9-b51f-17b97a6b7ecc service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:33.320464 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e769d8-1804-42c9-b51f-17b97a6b7ecc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:33.324995 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-56e769d8-1804-42c9-b51f-17b97a6b7ecc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:33.444512 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 880/1785] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:33 2022] PUT /identity/v3/projects/2afc430f1d89469bb55935ca6f1abfe6/users/064efa62b57c4a0c92ed9480aa0d6988/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:33.458066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc660ec7-c0c4-477d-b91d-1e2c09e3b72d None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:33.458620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc660ec7-c0c4-477d-b91d-1e2c09e3b72d None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:33.459052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc660ec7-c0c4-477d-b91d-1e2c09e3b72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:33.469924 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e769d8-1804-42c9-b51f-17b97a6b7ecc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:33.473277 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e769d8-1804-42c9-b51f-17b97a6b7ecc service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:33.504786 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 906/1786] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:45:33.520626 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-cc660ec7-c0c4-477d-b91d-1e2c09e3b72d None None] MFA Rules not processed for user `064efa62b57c4a0c92ed9480aa0d6988`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:33.522220 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-cc660ec7-c0c4-477d-b91d-1e2c09e3b72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:33.616963 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 881/1787] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:33.621223 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e769d8-1804-42c9-b51f-17b97a6b7ecc service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:33.635331 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e769d8-1804-42c9-b51f-17b97a6b7ecc service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:33.719427 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e6bd8-7de0-4a27-b8be-9e99c6679f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:33.720254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e6bd8-7de0-4a27-b8be-9e99c6679f65 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:33.720601 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e6bd8-7de0-4a27-b8be-9e99c6679f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:33.721581 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e6bd8-7de0-4a27-b8be-9e99c6679f65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:33.722302 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e6bd8-7de0-4a27-b8be-9e99c6679f65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:33.724332 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e6bd8-7de0-4a27-b8be-9e99c6679f65 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:33.825656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:33.833517 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 907/1788] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:33 2022] POST /identity/v3/projects => generated 375 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:45:33.837861 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc660ec7-c0c4-477d-b91d-1e2c09e3b72d None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:33.855467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc660ec7-c0c4-477d-b91d-1e2c09e3b72d None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:33.935935 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:34.000633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3fc47d-6032-41bb-a0a9-3840dada4596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:34.001811 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fc47d-6032-41bb-a0a9-3840dada4596 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:34.002336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fc47d-6032-41bb-a0a9-3840dada4596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:34.002740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fc47d-6032-41bb-a0a9-3840dada4596 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:34.003565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3fc47d-6032-41bb-a0a9-3840dada4596 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:34.010113 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3fc47d-6032-41bb-a0a9-3840dada4596 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:34.116026 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 882/1789] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:33 2022] POST /identity/v3/users => generated 288 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:34.117614 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e6bd8-7de0-4a27-b8be-9e99c6679f65 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:34.133524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e6bd8-7de0-4a27-b8be-9e99c6679f65 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:34.146708 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe507903-aa40-4ec9-864e-3788f4c4ad54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:34.146708 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe507903-aa40-4ec9-864e-3788f4c4ad54 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:34.146708 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe507903-aa40-4ec9-864e-3788f4c4ad54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:34.147829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe507903-aa40-4ec9-864e-3788f4c4ad54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:34.149400 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe507903-aa40-4ec9-864e-3788f4c4ad54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:34.151731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe507903-aa40-4ec9-864e-3788f4c4ad54 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:34.169813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 908/1790] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:34.174348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e7214-b801-4f26-9957-e160045811ee None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:34.175633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e7214-b801-4f26-9957-e160045811ee None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:34.176173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e7214-b801-4f26-9957-e160045811ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:34.186506 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe507903-aa40-4ec9-864e-3788f4c4ad54 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:34.200203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-2e4e7214-b801-4f26-9957-e160045811ee None None] MFA Rules not processed for user `1d7e431402f8433fb7565f69103485ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:34.202237 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-2e4e7214-b801-4f26-9957-e160045811ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:34.208123 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe507903-aa40-4ec9-864e-3788f4c4ad54 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:34.223673 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df456a-1181-49e9-a125-24b4e33a062c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:34.224363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df456a-1181-49e9-a125-24b4e33a062c admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:34.224699 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df456a-1181-49e9-a125-24b4e33a062c admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:34.224983 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df456a-1181-49e9-a125-24b4e33a062c admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:34.225722 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df456a-1181-49e9-a125-24b4e33a062c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:34.227328 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df456a-1181-49e9-a125-24b4e33a062c admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:34.259751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 909/1791] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:34.261247 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df456a-1181-49e9-a125-24b4e33a062c admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:34.294260 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df456a-1181-49e9-a125-24b4e33a062c admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:34.309082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d19ae-c9f9-430f-b897-6e1f44d588a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:34.309975 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d19ae-c9f9-430f-b897-6e1f44d588a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:34.310263 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d19ae-c9f9-430f-b897-6e1f44d588a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:34.310549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d19ae-c9f9-430f-b897-6e1f44d588a3 admin admin] PATH_INFO: `/v3/projects/f8cd7f34a24b4d51bd110db2c2249adf/users/21048f2d96134b5e9ae5612fadbf1b20/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:34.331901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 883/1792] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:34.347773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d19ae-c9f9-430f-b897-6e1f44d588a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8cd7f34a24b4d51bd110db2c2249adf, user_id=21048f2d96134b5e9ae5612fadbf1b20, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:34.350671 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d19ae-c9f9-430f-b897-6e1f44d588a3 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:34.499971 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 910/1793] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:34 2022] PUT /identity/v3/projects/f8cd7f34a24b4d51bd110db2c2249adf/users/21048f2d96134b5e9ae5612fadbf1b20/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:34.502645 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49164358-abd9-4b64-a849-1a7a3bf2486a None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:34.503077 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49164358-abd9-4b64-a849-1a7a3bf2486a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:34.503482 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49164358-abd9-4b64-a849-1a7a3bf2486a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:34.597108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-49164358-abd9-4b64-a849-1a7a3bf2486a None None] MFA Rules not processed for user `21048f2d96134b5e9ae5612fadbf1b20`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:34.598262 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-49164358-abd9-4b64-a849-1a7a3bf2486a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:34.767727 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 884/1794] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:34.771894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d19ae-c9f9-430f-b897-6e1f44d588a3 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:34.898075 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d19ae-c9f9-430f-b897-6e1f44d588a3 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:34.906580 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21f942-a7eb-4e2e-a01c-c794be25e14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:34.908123 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21f942-a7eb-4e2e-a01c-c794be25e14c admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:34.908933 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21f942-a7eb-4e2e-a01c-c794be25e14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:34.909429 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21f942-a7eb-4e2e-a01c-c794be25e14c admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:34.910787 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21f942-a7eb-4e2e-a01c-c794be25e14c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:34.914488 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21f942-a7eb-4e2e-a01c-c794be25e14c admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:34.986386 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:35.027948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 911/1795] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:34 2022] POST /identity/v3/projects => generated 375 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:35.032068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49164358-abd9-4b64-a849-1a7a3bf2486a None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:35.043593 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49164358-abd9-4b64-a849-1a7a3bf2486a None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:35.119478 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160f9da-f361-419a-933b-6ea36f28e742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:35.120192 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160f9da-f361-419a-933b-6ea36f28e742 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:35.122801 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160f9da-f361-419a-933b-6ea36f28e742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:35.122801 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160f9da-f361-419a-933b-6ea36f28e742 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:35.122801 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b160f9da-f361-419a-933b-6ea36f28e742 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:35.124003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b160f9da-f361-419a-933b-6ea36f28e742 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:35.233785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 885/1796] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:35 2022] POST /identity/v3/users => generated 294 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:35.237875 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21f942-a7eb-4e2e-a01c-c794be25e14c admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:35.250695 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21f942-a7eb-4e2e-a01c-c794be25e14c admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:35.257309 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867fe8e-55b9-491a-af4e-dd554c1c7fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:35.257993 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867fe8e-55b9-491a-af4e-dd554c1c7fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:35.258277 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867fe8e-55b9-491a-af4e-dd554c1c7fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:35.258715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867fe8e-55b9-491a-af4e-dd554c1c7fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:35.259524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867fe8e-55b9-491a-af4e-dd554c1c7fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:35.261257 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867fe8e-55b9-491a-af4e-dd554c1c7fd6 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:35.277402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 912/1797] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:45:35.278118 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160f9da-f361-419a-933b-6ea36f28e742 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:35.288899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160f9da-f361-419a-933b-6ea36f28e742 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:35.295353 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825319d7-f375-41ce-b292-d03a6c7a87e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:35.296331 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825319d7-f375-41ce-b292-d03a6c7a87e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:35.296721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825319d7-f375-41ce-b292-d03a6c7a87e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:35.297010 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825319d7-f375-41ce-b292-d03a6c7a87e7 admin admin] PATH_INFO: `/v3/projects/13f7d51259a7480fa9e90458c9f34e87/users/74c33b0fcdca45888f0220d01f6870f0/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:35.315293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825319d7-f375-41ce-b292-d03a6c7a87e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f7d51259a7480fa9e90458c9f34e87, user_id=74c33b0fcdca45888f0220d01f6870f0, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:35.316114 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825319d7-f375-41ce-b292-d03a6c7a87e7 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:35.418876 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 886/1798] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:35 2022] PUT /identity/v3/projects/13f7d51259a7480fa9e90458c9f34e87/users/74c33b0fcdca45888f0220d01f6870f0/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:45:35.420156 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867fe8e-55b9-491a-af4e-dd554c1c7fd6 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:35.435750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867fe8e-55b9-491a-af4e-dd554c1c7fd6 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:35.489164 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:35.560276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:35.570644 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a44201-f340-4ceb-a807-7734d86d858a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:35.571515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a44201-f340-4ceb-a807-7734d86d858a admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:35.571959 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a44201-f340-4ceb-a807-7734d86d858a admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:35.572366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a44201-f340-4ceb-a807-7734d86d858a admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:35.573869 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a44201-f340-4ceb-a807-7734d86d858a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:35.584359 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a44201-f340-4ceb-a807-7734d86d858a admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:35.587935 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 913/1799] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:45:35.592324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825319d7-f375-41ce-b292-d03a6c7a87e7 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:35.613048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825319d7-f375-41ce-b292-d03a6c7a87e7 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:35.630531 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9742d2-d977-4e0f-a06c-414e36dc11a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:35.630531 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9742d2-d977-4e0f-a06c-414e36dc11a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:35.630531 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9742d2-d977-4e0f-a06c-414e36dc11a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:35.630531 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9742d2-d977-4e0f-a06c-414e36dc11a1 admin admin] PATH_INFO: `/v3/projects/13f7d51259a7480fa9e90458c9f34e87/users/74c33b0fcdca45888f0220d01f6870f0/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:35.630813 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9742d2-d977-4e0f-a06c-414e36dc11a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f7d51259a7480fa9e90458c9f34e87, user_id=74c33b0fcdca45888f0220d01f6870f0, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:35.632236 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9742d2-d977-4e0f-a06c-414e36dc11a1 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:35.649748 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 887/1800] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:35 2022] PUT /identity/v3/projects/13f7d51259a7480fa9e90458c9f34e87/users/74c33b0fcdca45888f0220d01f6870f0/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:35.660294 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d831211-7332-40b7-be7b-de68d4d63993 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:35.660783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d831211-7332-40b7-be7b-de68d4d63993 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:35.661208 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d831211-7332-40b7-be7b-de68d4d63993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:35.713301 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-3d831211-7332-40b7-be7b-de68d4d63993 None None] MFA Rules not processed for user `74c33b0fcdca45888f0220d01f6870f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:35.718056 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3d831211-7332-40b7-be7b-de68d4d63993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:35.809262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 914/1801] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:35.820936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9742d2-d977-4e0f-a06c-414e36dc11a1 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:35.881768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:35.885955 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9742d2-d977-4e0f-a06c-414e36dc11a1 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:35.890248 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b26aa3-954b-4206-b364-85321a9ee483 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:35.891009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b26aa3-954b-4206-b364-85321a9ee483 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:35.891397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b26aa3-954b-4206-b364-85321a9ee483 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:35.891779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b26aa3-954b-4206-b364-85321a9ee483 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:35.954443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b26aa3-954b-4206-b364-85321a9ee483 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:35.955963 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b26aa3-954b-4206-b364-85321a9ee483 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:35.965460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 888/1802] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:37.761381 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d912d-7c3b-4138-b057-c296fdd440ee None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:37.762262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d912d-7c3b-4138-b057-c296fdd440ee None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:37.762765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d912d-7c3b-4138-b057-c296fdd440ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:37.794298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-433d912d-7c3b-4138-b057-c296fdd440ee None None] MFA Rules not processed for user `0a7c204a298c42f0bbc32fa9a6ce1f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:37.805847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-433d912d-7c3b-4138-b057-c296fdd440ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:37.927823 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 915/1803] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 22:45:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:38.423557 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b26aa3-954b-4206-b364-85321a9ee483 service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:38.423557 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-a9b26aa3-954b-4206-b364-85321a9ee483 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:38.458798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d912d-7c3b-4138-b057-c296fdd440ee None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:38.460506 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-433d912d-7c3b-4138-b057-c296fdd440ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:38.477757 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d912d-7c3b-4138-b057-c296fdd440ee None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:38.484807 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e152ed7-a487-4bd8-9685-81ba27123ee7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:38.485831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e152ed7-a487-4bd8-9685-81ba27123ee7 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:38.486328 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e152ed7-a487-4bd8-9685-81ba27123ee7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:38.486763 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e152ed7-a487-4bd8-9685-81ba27123ee7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:38.545413 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b26aa3-954b-4206-b364-85321a9ee483 service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:38.555307 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a38f23-3bac-4b7c-b2da-217f03149172 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b321586cdb3d4d25904472aaa4adba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:38.556189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a38f23-3bac-4b7c-b2da-217f03149172 service swift] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:38.556603 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a38f23-3bac-4b7c-b2da-217f03149172 service swift] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:38.556969 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a38f23-3bac-4b7c-b2da-217f03149172 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:38.568778 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e152ed7-a487-4bd8-9685-81ba27123ee7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:38.570918 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e152ed7-a487-4bd8-9685-81ba27123ee7 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:38.571785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a38f23-3bac-4b7c-b2da-217f03149172 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:38.574562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a38f23-3bac-4b7c-b2da-217f03149172 service swift] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:38.586620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 889/1804] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 22:45:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:45:38.595741 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 916/1805] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:44.257454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a38f23-3bac-4b7c-b2da-217f03149172 service swift] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:44.266995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a38f23-3bac-4b7c-b2da-217f03149172 service swift] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:44.273498 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca651caf-4569-4010-8e44-d0e78cf31b9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:44.273498 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca651caf-4569-4010-8e44-d0e78cf31b9e service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:44.273498 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca651caf-4569-4010-8e44-d0e78cf31b9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:44.273498 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca651caf-4569-4010-8e44-d0e78cf31b9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:44.392914 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca651caf-4569-4010-8e44-d0e78cf31b9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:44.394528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca651caf-4569-4010-8e44-d0e78cf31b9e service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:44.408379 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 890/1806] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:45:44.701110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e152ed7-a487-4bd8-9685-81ba27123ee7 service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:44.709978 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e152ed7-a487-4bd8-9685-81ba27123ee7 service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:44.714959 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c3e85-53c1-46c8-8617-dd096105f926 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:44.715878 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c3e85-53c1-46c8-8617-dd096105f926 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:44.716785 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c3e85-53c1-46c8-8617-dd096105f926 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:44.717309 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c3e85-53c1-46c8-8617-dd096105f926 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:44.730220 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c3e85-53c1-46c8-8617-dd096105f926 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:44.731937 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c3e85-53c1-46c8-8617-dd096105f926 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:44.746024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 917/1807] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:51.479049 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca651caf-4569-4010-8e44-d0e78cf31b9e service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:51.480446 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ca651caf-4569-4010-8e44-d0e78cf31b9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:51.559271 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:51.600124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca651caf-4569-4010-8e44-d0e78cf31b9e service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:51.606407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca711ef-e895-4ac3-af82-0ca838c9ce2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:51.606984 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca711ef-e895-4ac3-af82-0ca838c9ce2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:51.607274 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca711ef-e895-4ac3-af82-0ca838c9ce2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:51.607553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca711ef-e895-4ac3-af82-0ca838c9ce2f admin admin] PATH_INFO: `/v3/users/944825f6758949288386555fffd7dc07` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:51.609031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca711ef-e895-4ac3-af82-0ca838c9ce2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=944825f6758949288386555fffd7dc07)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:51.610796 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca711ef-e895-4ac3-af82-0ca838c9ce2f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:51.845126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-dca711ef-e895-4ac3-af82-0ca838c9ce2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '944825f6758949288386555fffd7dc07'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:51.880317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-dca711ef-e895-4ac3-af82-0ca838c9ce2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '944825f6758949288386555fffd7dc07'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:51.937326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ff432-8ab6-4fc7-b882-c22e004d5189 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:51.937326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ff432-8ab6-4fc7-b882-c22e004d5189 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:51.937326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ff432-8ab6-4fc7-b882-c22e004d5189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:51.939675 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:51.956650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-dca711ef-e895-4ac3-af82-0ca838c9ce2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '944825f6758949288386555fffd7dc07'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:51.957402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-dca711ef-e895-4ac3-af82-0ca838c9ce2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '944825f6758949288386555fffd7dc07'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:51.958127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-dca711ef-e895-4ac3-af82-0ca838c9ce2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '944825f6758949288386555fffd7dc07'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:51.996059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-3c0ff432-8ab6-4fc7-b882-c22e004d5189 None None] MFA Rules not processed for user `ec7412a9a3dc46369fe7b4771e3437de`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:51.998280 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3c0ff432-8ab6-4fc7-b882-c22e004d5189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:52.005913 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 891/1808] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:45:51 2022] DELETE /identity/v3/users/944825f6758949288386555fffd7dc07 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:52.012171 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67a235-b169-4de7-b80b-e3a8e0835605 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:52.012306 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67a235-b169-4de7-b80b-e3a8e0835605 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:52.012631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67a235-b169-4de7-b80b-e3a8e0835605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:52.057067 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-eb67a235-b169-4de7-b80b-e3a8e0835605 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:52.059475 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-eb67a235-b169-4de7-b80b-e3a8e0835605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:52.102281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 918/1809] 10.222.0.13 () {58 vars in 1015 bytes} [Thu Jun 23 22:45:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:45:52.135592 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 892/1810] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:52.145595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ff432-8ab6-4fc7-b882-c22e004d5189 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:52.147129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3c0ff432-8ab6-4fc7-b882-c22e004d5189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:52.158483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ff432-8ab6-4fc7-b882-c22e004d5189 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:52.162400 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8b588-fae9-4d63-aed8-7c3d891f2763 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:52.163105 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8b588-fae9-4d63-aed8-7c3d891f2763 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:52.163343 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8b588-fae9-4d63-aed8-7c3d891f2763 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:52.163627 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8b588-fae9-4d63-aed8-7c3d891f2763 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:52.182804 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8b588-fae9-4d63-aed8-7c3d891f2763 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:52.184468 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8b588-fae9-4d63-aed8-7c3d891f2763 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:52.195097 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 919/1811] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:52.378077 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67a235-b169-4de7-b80b-e3a8e0835605 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:52.380208 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-eb67a235-b169-4de7-b80b-e3a8e0835605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:52.382633 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8b588-fae9-4d63-aed8-7c3d891f2763 service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:52.384383 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e8e8b588-fae9-4d63-aed8-7c3d891f2763 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:52.393599 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67a235-b169-4de7-b80b-e3a8e0835605 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:52.398260 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59739c-08cf-4f11-b42f-ac131695c39a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:52.398897 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8b588-fae9-4d63-aed8-7c3d891f2763 service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:52.399994 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59739c-08cf-4f11-b42f-ac131695c39a service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:52.401122 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59739c-08cf-4f11-b42f-ac131695c39a service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:52.401660 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59739c-08cf-4f11-b42f-ac131695c39a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:52.404664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d917778-fa3b-4f02-b086-e9fa41523b91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:52.406257 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d917778-fa3b-4f02-b086-e9fa41523b91 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:52.406257 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d917778-fa3b-4f02-b086-e9fa41523b91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:52.406257 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d917778-fa3b-4f02-b086-e9fa41523b91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:52.410433 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59739c-08cf-4f11-b42f-ac131695c39a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:52.412106 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59739c-08cf-4f11-b42f-ac131695c39a service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:52.416703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d917778-fa3b-4f02-b086-e9fa41523b91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:52.418644 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d917778-fa3b-4f02-b086-e9fa41523b91 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:52.423294 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 893/1812] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:52.423984 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59739c-08cf-4f11-b42f-ac131695c39a service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:52.429644 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 920/1813] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:52.438373 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59739c-08cf-4f11-b42f-ac131695c39a service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:52.444434 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90428c-ee41-47fa-8814-c65c1e5d9d36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:52.445667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90428c-ee41-47fa-8814-c65c1e5d9d36 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:52.446687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90428c-ee41-47fa-8814-c65c1e5d9d36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:52.446687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90428c-ee41-47fa-8814-c65c1e5d9d36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:52.458106 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90428c-ee41-47fa-8814-c65c1e5d9d36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:52.473204 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90428c-ee41-47fa-8814-c65c1e5d9d36 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:52.491393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 894/1814] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:53.432508 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d917778-fa3b-4f02-b086-e9fa41523b91 service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:53.434091 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3d917778-fa3b-4f02-b086-e9fa41523b91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:53.457222 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d917778-fa3b-4f02-b086-e9fa41523b91 service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:53.464134 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0558b4f-01ab-4fb9-a93a-4d2f76e2deee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:53.465416 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0558b4f-01ab-4fb9-a93a-4d2f76e2deee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:53.465886 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0558b4f-01ab-4fb9-a93a-4d2f76e2deee admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:53.466499 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0558b4f-01ab-4fb9-a93a-4d2f76e2deee admin admin] PATH_INFO: `/v3/users/064efa62b57c4a0c92ed9480aa0d6988` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:53.468211 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0558b4f-01ab-4fb9-a93a-4d2f76e2deee admin admin] RBAC: Authorizing `identity:delete_user(user_id=064efa62b57c4a0c92ed9480aa0d6988)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:53.471857 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0558b4f-01ab-4fb9-a93a-4d2f76e2deee admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:53.670257 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90428c-ee41-47fa-8814-c65c1e5d9d36 service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:53.672239 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3c90428c-ee41-47fa-8814-c65c1e5d9d36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:53.692413 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90428c-ee41-47fa-8814-c65c1e5d9d36 service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:53.700860 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f85134-9c7c-4fc7-ad5b-861aded90e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:53.702055 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f85134-9c7c-4fc7-ad5b-861aded90e9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:53.702838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f85134-9c7c-4fc7-ad5b-861aded90e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:53.703388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f85134-9c7c-4fc7-ad5b-861aded90e9b admin admin] PATH_INFO: `/v3/projects/3b618789981848078fb57f45ea89d581` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:53.705503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f85134-9c7c-4fc7-ad5b-861aded90e9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b618789981848078fb57f45ea89d581)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:53.708352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f85134-9c7c-4fc7-ad5b-861aded90e9b admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:53.732052 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-f0558b4f-01ab-4fb9-a93a-4d2f76e2deee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '064efa62b57c4a0c92ed9480aa0d6988'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:53.736791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:53.768644 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-f0558b4f-01ab-4fb9-a93a-4d2f76e2deee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '064efa62b57c4a0c92ed9480aa0d6988'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:53.769576 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-f0558b4f-01ab-4fb9-a93a-4d2f76e2deee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '064efa62b57c4a0c92ed9480aa0d6988'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:53.770364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-f0558b4f-01ab-4fb9-a93a-4d2f76e2deee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '064efa62b57c4a0c92ed9480aa0d6988'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:53.781720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-f0558b4f-01ab-4fb9-a93a-4d2f76e2deee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '064efa62b57c4a0c92ed9480aa0d6988'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:53.865826 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-04f85134-9c7c-4fc7-ad5b-861aded90e9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b618789981848078fb57f45ea89d581', 'request_id': 'req-04f85134-9c7c-4fc7-ad5b-861aded90e9b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:53.871654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 921/1815] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:45:53 2022] DELETE /identity/v3/users/064efa62b57c4a0c92ed9480aa0d6988 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:53.873559 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1c60c-c670-4e3c-8a15-ca006328d50f None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:53.874056 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1c60c-c670-4e3c-8a15-ca006328d50f None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:53.874411 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1c60c-c670-4e3c-8a15-ca006328d50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:53.913992 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-05a1c60c-c670-4e3c-8a15-ca006328d50f None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:53.917395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-05a1c60c-c670-4e3c-8a15-ca006328d50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:53.980136 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-04f85134-9c7c-4fc7-ad5b-861aded90e9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b618789981848078fb57f45ea89d581', 'request_id': 'req-04f85134-9c7c-4fc7-ad5b-861aded90e9b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:53.988565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-04f85134-9c7c-4fc7-ad5b-861aded90e9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b618789981848078fb57f45ea89d581', 'request_id': 'req-04f85134-9c7c-4fc7-ad5b-861aded90e9b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:54.012752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-04f85134-9c7c-4fc7-ad5b-861aded90e9b admin admin] The token cache is being invalidate because project 3b618789981848078fb57f45ea89d581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:45:54.014404 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-04f85134-9c7c-4fc7-ad5b-861aded90e9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:54.016783 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-04f85134-9c7c-4fc7-ad5b-861aded90e9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:54.018964 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 895/1816] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:45:53 2022] DELETE /identity/v3/projects/3b618789981848078fb57f45ea89d581 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:45:54.021355 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:54.023063 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f85134-9c7c-4fc7-ad5b-861aded90e9b admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:54.026378 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-04f85134-9c7c-4fc7-ad5b-861aded90e9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:54.036033 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 922/1817] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:54.069879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1c60c-c670-4e3c-8a15-ca006328d50f None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:54.073567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-05a1c60c-c670-4e3c-8a15-ca006328d50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:54.125864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f85134-9c7c-4fc7-ad5b-861aded90e9b admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:54.132008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390de2ab-0deb-4579-8c41-ad4549c803f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:54.133113 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390de2ab-0deb-4579-8c41-ad4549c803f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:54.133387 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390de2ab-0deb-4579-8c41-ad4549c803f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:54.133666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390de2ab-0deb-4579-8c41-ad4549c803f9 admin admin] PATH_INFO: `/v3/users/76ead827916f4f048a334f35df362fcf` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:54.136492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390de2ab-0deb-4579-8c41-ad4549c803f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76ead827916f4f048a334f35df362fcf)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:54.139144 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390de2ab-0deb-4579-8c41-ad4549c803f9 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:54.162384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1c60c-c670-4e3c-8a15-ca006328d50f None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:54.169998 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b343e-7ea1-421c-a29e-9fccc11ad9f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:54.170276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b343e-7ea1-421c-a29e-9fccc11ad9f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:54.170574 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b343e-7ea1-421c-a29e-9fccc11ad9f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:54.170861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b343e-7ea1-421c-a29e-9fccc11ad9f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:54.183742 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b343e-7ea1-421c-a29e-9fccc11ad9f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:54.185292 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b343e-7ea1-421c-a29e-9fccc11ad9f2 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:54.198668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 923/1818] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:45:54.449772 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-390de2ab-0deb-4579-8c41-ad4549c803f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76ead827916f4f048a334f35df362fcf'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:54.483833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-390de2ab-0deb-4579-8c41-ad4549c803f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76ead827916f4f048a334f35df362fcf'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:54.573686 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-390de2ab-0deb-4579-8c41-ad4549c803f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76ead827916f4f048a334f35df362fcf'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:54.585458 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-390de2ab-0deb-4579-8c41-ad4549c803f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76ead827916f4f048a334f35df362fcf'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:54.585458 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-390de2ab-0deb-4579-8c41-ad4549c803f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76ead827916f4f048a334f35df362fcf'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:54.614137 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 896/1819] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:45:54 2022] DELETE /identity/v3/users/76ead827916f4f048a334f35df362fcf => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:54.799804 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752cd29-fc4f-4f7c-9511-63720d2e09c0 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:54.800382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752cd29-fc4f-4f7c-9511-63720d2e09c0 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:54.800701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752cd29-fc4f-4f7c-9511-63720d2e09c0 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:54.802238 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 924/1820] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 22:45:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:45:54.820959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c4cd2-8bda-4163-9548-104655987ba7 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:54.822627 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c4cd2-8bda-4163-9548-104655987ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:54.823288 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c4cd2-8bda-4163-9548-104655987ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:54.844848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-527c4cd2-8bda-4163-9548-104655987ba7 None None] MFA Rules not processed for user `159cc8befc3a4d57bbd313832084db62`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:54.846134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-527c4cd2-8bda-4163-9548-104655987ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:54.878505 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:54.946848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 897/1821] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 22:45:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:54.972701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752cd29-fc4f-4f7c-9511-63720d2e09c0 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:54.974681 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d752cd29-fc4f-4f7c-9511-63720d2e09c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:55.015811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752cd29-fc4f-4f7c-9511-63720d2e09c0 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:55.022975 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b04bd0-9848-4b9d-93ed-a3dff98926c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:55.023804 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b04bd0-9848-4b9d-93ed-a3dff98926c2 service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:55.024161 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b04bd0-9848-4b9d-93ed-a3dff98926c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:55.024558 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b04bd0-9848-4b9d-93ed-a3dff98926c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:55.034507 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b04bd0-9848-4b9d-93ed-a3dff98926c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:55.036220 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b04bd0-9848-4b9d-93ed-a3dff98926c2 service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:55.049330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 925/1822] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:55.445612 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3871ac5-bf94-44ef-ab24-5fb4a6b74b4a None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:55.446417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3871ac5-bf94-44ef-ab24-5fb4a6b74b4a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:55.447477 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3871ac5-bf94-44ef-ab24-5fb4a6b74b4a None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:55.450325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 898/1823] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 22:45:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:45:55.464207 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b86c48-9709-460f-b6c3-634e66ca1111 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:55.464822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b86c48-9709-460f-b6c3-634e66ca1111 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:55.465323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b86c48-9709-460f-b6c3-634e66ca1111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:55.522918 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-16b86c48-9709-460f-b6c3-634e66ca1111 None None] MFA Rules not processed for user `159cc8befc3a4d57bbd313832084db62`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:55.525863 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-16b86c48-9709-460f-b6c3-634e66ca1111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:55.545400 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 926/1824] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 22:45:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:55.564938 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3871ac5-bf94-44ef-ab24-5fb4a6b74b4a None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:55.591430 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3871ac5-bf94-44ef-ab24-5fb4a6b74b4a None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:55.604499 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a0242-9a67-4511-a21a-65808c5f2790 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:55.605384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a0242-9a67-4511-a21a-65808c5f2790 service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:55.605886 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a0242-9a67-4511-a21a-65808c5f2790 service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:55.606241 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a0242-9a67-4511-a21a-65808c5f2790 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:55.620804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a0242-9a67-4511-a21a-65808c5f2790 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:55.625884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a0242-9a67-4511-a21a-65808c5f2790 service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:55.640529 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 899/1825] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:55.727098 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b86c48-9709-460f-b6c3-634e66ca1111 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:55.729122 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-16b86c48-9709-460f-b6c3-634e66ca1111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:55.733669 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a0242-9a67-4511-a21a-65808c5f2790 service nova] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:55.734942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-1f4a0242-9a67-4511-a21a-65808c5f2790 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:55.843218 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a0242-9a67-4511-a21a-65808c5f2790 service nova] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:55.847076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b86c48-9709-460f-b6c3-634e66ca1111 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:55.849669 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a3a22-8769-4cab-afd7-126ef69ab9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:55.850957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a3a22-8769-4cab-afd7-126ef69ab9d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:55.851342 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a3a22-8769-4cab-afd7-126ef69ab9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:55.851738 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a3a22-8769-4cab-afd7-126ef69ab9d0 admin admin] PATH_INFO: `/v3/projects/2afc430f1d89469bb55935ca6f1abfe6` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:55.853250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a3a22-8769-4cab-afd7-126ef69ab9d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2afc430f1d89469bb55935ca6f1abfe6)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:55.853774 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d222849-48b7-481e-8ee4-5a3dbe1d2207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:55.855067 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d222849-48b7-481e-8ee4-5a3dbe1d2207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:55.855363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d222849-48b7-481e-8ee4-5a3dbe1d2207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:55.855466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a3a22-8769-4cab-afd7-126ef69ab9d0 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:55.855646 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d222849-48b7-481e-8ee4-5a3dbe1d2207 admin admin] PATH_INFO: `/v3/projects/13acdb0a10a2448f81e804fc41c1d1af` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:55.857245 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d222849-48b7-481e-8ee4-5a3dbe1d2207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13acdb0a10a2448f81e804fc41c1d1af)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:55.859426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d222849-48b7-481e-8ee4-5a3dbe1d2207 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:56.010485 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-896a3a22-8769-4cab-afd7-126ef69ab9d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2afc430f1d89469bb55935ca6f1abfe6', 'request_id': 'req-896a3a22-8769-4cab-afd7-126ef69ab9d0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:56.019945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2d222849-48b7-481e-8ee4-5a3dbe1d2207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13acdb0a10a2448f81e804fc41c1d1af', 'request_id': 'req-2d222849-48b7-481e-8ee4-5a3dbe1d2207'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:56.034950 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2d222849-48b7-481e-8ee4-5a3dbe1d2207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13acdb0a10a2448f81e804fc41c1d1af', 'request_id': 'req-2d222849-48b7-481e-8ee4-5a3dbe1d2207'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:56.037933 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2d222849-48b7-481e-8ee4-5a3dbe1d2207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13acdb0a10a2448f81e804fc41c1d1af', 'request_id': 'req-2d222849-48b7-481e-8ee4-5a3dbe1d2207'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:56.082728 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-896a3a22-8769-4cab-afd7-126ef69ab9d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2afc430f1d89469bb55935ca6f1abfe6', 'request_id': 'req-896a3a22-8769-4cab-afd7-126ef69ab9d0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:56.094872 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-896a3a22-8769-4cab-afd7-126ef69ab9d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2afc430f1d89469bb55935ca6f1abfe6', 'request_id': 'req-896a3a22-8769-4cab-afd7-126ef69ab9d0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:56.126941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-896a3a22-8769-4cab-afd7-126ef69ab9d0 admin admin] The token cache is being invalidate because project 2afc430f1d89469bb55935ca6f1abfe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:45:56.127497 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-896a3a22-8769-4cab-afd7-126ef69ab9d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:56.129090 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-896a3a22-8769-4cab-afd7-126ef69ab9d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:56.131117 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 900/1826] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:45:55 2022] DELETE /identity/v3/projects/2afc430f1d89469bb55935ca6f1abfe6 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:45:56.135582 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a3a22-8769-4cab-afd7-126ef69ab9d0 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:56.137274 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-896a3a22-8769-4cab-afd7-126ef69ab9d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:56.155484 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2d222849-48b7-481e-8ee4-5a3dbe1d2207 admin admin] The token cache is being invalidate because project 13acdb0a10a2448f81e804fc41c1d1af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:45:56.155484 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2d222849-48b7-481e-8ee4-5a3dbe1d2207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:56.156622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2d222849-48b7-481e-8ee4-5a3dbe1d2207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:56.167392 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 927/1827] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:45:55 2022] DELETE /identity/v3/projects/13acdb0a10a2448f81e804fc41c1d1af => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:56.169285 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5992012-9350-4589-9f55-74dfba1e81a0 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:56.169735 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5992012-9350-4589-9f55-74dfba1e81a0 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:56.170200 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5992012-9350-4589-9f55-74dfba1e81a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:56.195668 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:56.204800 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-e5992012-9350-4589-9f55-74dfba1e81a0 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:56.206027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e5992012-9350-4589-9f55-74dfba1e81a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:56.219657 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:56.243571 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a3a22-8769-4cab-afd7-126ef69ab9d0 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:56.316657 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 928/1828] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:56.317079 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d7159-f04a-4791-8ab9-52110812a939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:56.318239 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d7159-f04a-4791-8ab9-52110812a939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:56.318700 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d7159-f04a-4791-8ab9-52110812a939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:56.319132 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d7159-f04a-4791-8ab9-52110812a939 admin admin] PATH_INFO: `/v3/users/21048f2d96134b5e9ae5612fadbf1b20` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:56.320549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d7159-f04a-4791-8ab9-52110812a939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21048f2d96134b5e9ae5612fadbf1b20)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:56.323170 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d7159-f04a-4791-8ab9-52110812a939 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:56.325030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cc435-c2a6-4fd7-8848-6b9ed451d7a7 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:56.325327 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cc435-c2a6-4fd7-8848-6b9ed451d7a7 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:56.325612 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cc435-c2a6-4fd7-8848-6b9ed451d7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:56.355592 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-646cc435-c2a6-4fd7-8848-6b9ed451d7a7 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:56.356852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-646cc435-c2a6-4fd7-8848-6b9ed451d7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:56.365890 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 929/1829] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:56.371197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cc435-c2a6-4fd7-8848-6b9ed451d7a7 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:56.387832 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cc435-c2a6-4fd7-8848-6b9ed451d7a7 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:56.392772 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db0b70-1080-4ec4-a125-eacb929b6d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:56.393451 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db0b70-1080-4ec4-a125-eacb929b6d25 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:56.393739 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db0b70-1080-4ec4-a125-eacb929b6d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:56.394022 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db0b70-1080-4ec4-a125-eacb929b6d25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:56.394935 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db0b70-1080-4ec4-a125-eacb929b6d25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:45:56.395155 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db0b70-1080-4ec4-a125-eacb929b6d25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:56.397313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db0b70-1080-4ec4-a125-eacb929b6d25 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:56.410540 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 930/1830] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:45:56.414707 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db0b70-1080-4ec4-a125-eacb929b6d25 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:56.425798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db0b70-1080-4ec4-a125-eacb929b6d25 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:56.432373 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056e00c-65f3-4f00-ba47-4c2d5b4a5a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:56.433191 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056e00c-65f3-4f00-ba47-4c2d5b4a5a37 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:56.433491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056e00c-65f3-4f00-ba47-4c2d5b4a5a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:56.433778 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056e00c-65f3-4f00-ba47-4c2d5b4a5a37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:56.434485 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056e00c-65f3-4f00-ba47-4c2d5b4a5a37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:56.436504 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056e00c-65f3-4f00-ba47-4c2d5b4a5a37 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:56.501790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-773d7159-f04a-4791-8ab9-52110812a939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21048f2d96134b5e9ae5612fadbf1b20'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:56.515202 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-773d7159-f04a-4791-8ab9-52110812a939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21048f2d96134b5e9ae5612fadbf1b20'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:56.525697 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 931/1831] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:56 2022] POST /identity/v3/projects => generated 355 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:45:56.531825 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056e00c-65f3-4f00-ba47-4c2d5b4a5a37 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:56.591709 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056e00c-65f3-4f00-ba47-4c2d5b4a5a37 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:56.595851 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-773d7159-f04a-4791-8ab9-52110812a939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21048f2d96134b5e9ae5612fadbf1b20'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:56.596505 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-773d7159-f04a-4791-8ab9-52110812a939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21048f2d96134b5e9ae5612fadbf1b20'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:56.596664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103d2e3-8727-447d-b3f1-bf88f9d926fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:56.597167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-773d7159-f04a-4791-8ab9-52110812a939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21048f2d96134b5e9ae5612fadbf1b20'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:56.597278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103d2e3-8727-447d-b3f1-bf88f9d926fe admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:56.597490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103d2e3-8727-447d-b3f1-bf88f9d926fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:56.597772 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103d2e3-8727-447d-b3f1-bf88f9d926fe admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:56.598494 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103d2e3-8727-447d-b3f1-bf88f9d926fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:56.600462 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103d2e3-8727-447d-b3f1-bf88f9d926fe admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:56.610604 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 901/1832] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:45:56 2022] DELETE /identity/v3/users/21048f2d96134b5e9ae5612fadbf1b20 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:56.674533 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 932/1833] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:56 2022] POST /identity/v3/users => generated 278 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:45:56.675723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d7159-f04a-4791-8ab9-52110812a939 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:56.677533 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-773d7159-f04a-4791-8ab9-52110812a939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:56.746188 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:56.750320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d7159-f04a-4791-8ab9-52110812a939 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:56.755270 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0fb5f-19c9-43eb-a23d-0e386ecfde14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:56.755940 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0fb5f-19c9-43eb-a23d-0e386ecfde14 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:56.756225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0fb5f-19c9-43eb-a23d-0e386ecfde14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:56.756506 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0fb5f-19c9-43eb-a23d-0e386ecfde14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:56.757352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0fb5f-19c9-43eb-a23d-0e386ecfde14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:56.758918 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0fb5f-19c9-43eb-a23d-0e386ecfde14 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:56.769745 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 902/1834] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:56.773813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103d2e3-8727-447d-b3f1-bf88f9d926fe admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:56.784332 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103d2e3-8727-447d-b3f1-bf88f9d926fe admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:56.789334 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5779a388-d3ee-46d3-b7ef-888f42ad4cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:56.790114 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779a388-d3ee-46d3-b7ef-888f42ad4cae admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:56.790472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779a388-d3ee-46d3-b7ef-888f42ad4cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:56.790758 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779a388-d3ee-46d3-b7ef-888f42ad4cae admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:56.791652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5779a388-d3ee-46d3-b7ef-888f42ad4cae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:56.793384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5779a388-d3ee-46d3-b7ef-888f42ad4cae admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:56.817341 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 933/1835] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:56.821687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0fb5f-19c9-43eb-a23d-0e386ecfde14 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:56.833176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0fb5f-19c9-43eb-a23d-0e386ecfde14 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:56.838371 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb1a82-c8d4-400e-8a92-91417cf63aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:56.839343 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb1a82-c8d4-400e-8a92-91417cf63aa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:56.840098 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb1a82-c8d4-400e-8a92-91417cf63aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:56.840098 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb1a82-c8d4-400e-8a92-91417cf63aa0 admin admin] PATH_INFO: `/v3/projects/b8a97d33db024a47b8c9862e53d14664/users/fbdbbeecfe52453cae35029faa599153/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:56.854675 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb1a82-c8d4-400e-8a92-91417cf63aa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8a97d33db024a47b8c9862e53d14664, user_id=fbdbbeecfe52453cae35029faa599153, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:56.857390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb1a82-c8d4-400e-8a92-91417cf63aa0 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:56.922530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 903/1836] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:56 2022] PUT /identity/v3/projects/b8a97d33db024a47b8c9862e53d14664/users/fbdbbeecfe52453cae35029faa599153/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:56.934558 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1331850-dc75-41d2-a62a-c75acdd47f7f None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:56.935026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1331850-dc75-41d2-a62a-c75acdd47f7f None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:56.935182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1331850-dc75-41d2-a62a-c75acdd47f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:56.963189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-d1331850-dc75-41d2-a62a-c75acdd47f7f None None] MFA Rules not processed for user `fbdbbeecfe52453cae35029faa599153`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:56.964377 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d1331850-dc75-41d2-a62a-c75acdd47f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:57.000750 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb1a82-c8d4-400e-8a92-91417cf63aa0 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:57.002719 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-52bb1a82-c8d4-400e-8a92-91417cf63aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:57.035978 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:57.079419 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 934/1837] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:56 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:57.083626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1331850-dc75-41d2-a62a-c75acdd47f7f None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:57.174276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb1a82-c8d4-400e-8a92-91417cf63aa0 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:57.176804 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1331850-dc75-41d2-a62a-c75acdd47f7f None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:57.181552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e7dd7-9bef-4d58-83fd-dc02f10b2cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:57.181823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce2187-32af-4132-a9fc-d9a3c6b067ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:57.183093 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e7dd7-9bef-4d58-83fd-dc02f10b2cba admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:57.183416 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e7dd7-9bef-4d58-83fd-dc02f10b2cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:57.183706 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e7dd7-9bef-4d58-83fd-dc02f10b2cba admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:57.183819 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce2187-32af-4132-a9fc-d9a3c6b067ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:57.184267 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce2187-32af-4132-a9fc-d9a3c6b067ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:57.184411 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e7dd7-9bef-4d58-83fd-dc02f10b2cba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:57.185150 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce2187-32af-4132-a9fc-d9a3c6b067ea admin admin] PATH_INFO: `/v3/projects/f8cd7f34a24b4d51bd110db2c2249adf` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:57.186726 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e7dd7-9bef-4d58-83fd-dc02f10b2cba admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:57.186955 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce2187-32af-4132-a9fc-d9a3c6b067ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8cd7f34a24b4d51bd110db2c2249adf)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:57.189688 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce2187-32af-4132-a9fc-d9a3c6b067ea admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:57.292740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 935/1838] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:57 2022] POST /identity/v3/projects => generated 355 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:57.297160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e7dd7-9bef-4d58-83fd-dc02f10b2cba admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:57.336928 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-4cce2187-32af-4132-a9fc-d9a3c6b067ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8cd7f34a24b4d51bd110db2c2249adf', 'request_id': 'req-4cce2187-32af-4132-a9fc-d9a3c6b067ea'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:57.383076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e7dd7-9bef-4d58-83fd-dc02f10b2cba admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:57.420404 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-4cce2187-32af-4132-a9fc-d9a3c6b067ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8cd7f34a24b4d51bd110db2c2249adf', 'request_id': 'req-4cce2187-32af-4132-a9fc-d9a3c6b067ea'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:57.429238 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-4cce2187-32af-4132-a9fc-d9a3c6b067ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8cd7f34a24b4d51bd110db2c2249adf', 'request_id': 'req-4cce2187-32af-4132-a9fc-d9a3c6b067ea'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:57.449434 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-4cce2187-32af-4132-a9fc-d9a3c6b067ea admin admin] The token cache is being invalidate because project f8cd7f34a24b4d51bd110db2c2249adf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:45:57.451638 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-4cce2187-32af-4132-a9fc-d9a3c6b067ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:57.455341 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-4cce2187-32af-4132-a9fc-d9a3c6b067ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:57.461765 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 904/1839] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:45:56 2022] DELETE /identity/v3/projects/f8cd7f34a24b4d51bd110db2c2249adf => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:57.473118 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce2187-32af-4132-a9fc-d9a3c6b067ea admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:57.476979 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-4cce2187-32af-4132-a9fc-d9a3c6b067ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:57.529822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b05c59-aa18-494a-8112-165242cc52f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:57.530616 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b05c59-aa18-494a-8112-165242cc52f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:57.530975 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b05c59-aa18-494a-8112-165242cc52f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:57.531319 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b05c59-aa18-494a-8112-165242cc52f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:57.532374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b05c59-aa18-494a-8112-165242cc52f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:57.536323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b05c59-aa18-494a-8112-165242cc52f3 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:57.601959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce2187-32af-4132-a9fc-d9a3c6b067ea admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:57.613321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147d50c-b410-4633-8d6b-5b465dd92f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:57.615323 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147d50c-b410-4633-8d6b-5b465dd92f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:57.616656 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147d50c-b410-4633-8d6b-5b465dd92f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:57.617203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147d50c-b410-4633-8d6b-5b465dd92f07 admin admin] PATH_INFO: `/v3/users/74c33b0fcdca45888f0220d01f6870f0` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:57.622828 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5147d50c-b410-4633-8d6b-5b465dd92f07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74c33b0fcdca45888f0220d01f6870f0)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:57.627665 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5147d50c-b410-4633-8d6b-5b465dd92f07 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:57.631619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:57.650796 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:57.660752 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 936/1840] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:57 2022] POST /identity/v3/users => generated 284 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:45:57.661718 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b05c59-aa18-494a-8112-165242cc52f3 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:57.663568 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d9b05c59-aa18-494a-8112-165242cc52f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:57.677379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b05c59-aa18-494a-8112-165242cc52f3 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:57.683887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c95ad4-bc29-4e23-8c07-4ab069723d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:57.685043 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c95ad4-bc29-4e23-8c07-4ab069723d58 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:57.685473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c95ad4-bc29-4e23-8c07-4ab069723d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:57.687477 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c95ad4-bc29-4e23-8c07-4ab069723d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:57.688416 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c95ad4-bc29-4e23-8c07-4ab069723d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:57.690307 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c95ad4-bc29-4e23-8c07-4ab069723d58 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:57.706402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 937/1841] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:45:57.711022 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c95ad4-bc29-4e23-8c07-4ab069723d58 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:57.722374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c95ad4-bc29-4e23-8c07-4ab069723d58 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:57.735393 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa335d-3aa9-486f-8d30-e7bae812b959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:57.744269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa335d-3aa9-486f-8d30-e7bae812b959 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:57.744269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa335d-3aa9-486f-8d30-e7bae812b959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:57.744269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa335d-3aa9-486f-8d30-e7bae812b959 admin admin] PATH_INFO: `/v3/projects/94491c37393f444880c6456030ae19c9/users/a7ed52bb1aba48f1a0454fcb7a97c0a8/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:57.764119 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa335d-3aa9-486f-8d30-e7bae812b959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94491c37393f444880c6456030ae19c9, user_id=a7ed52bb1aba48f1a0454fcb7a97c0a8, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:57.767823 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa335d-3aa9-486f-8d30-e7bae812b959 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:57.853545 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-5147d50c-b410-4633-8d6b-5b465dd92f07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74c33b0fcdca45888f0220d01f6870f0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:57.867314 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 938/1842] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:57 2022] PUT /identity/v3/projects/94491c37393f444880c6456030ae19c9/users/a7ed52bb1aba48f1a0454fcb7a97c0a8/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:57.871840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa335d-3aa9-486f-8d30-e7bae812b959 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:57.881912 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-5147d50c-b410-4633-8d6b-5b465dd92f07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74c33b0fcdca45888f0220d01f6870f0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:57.967763 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa335d-3aa9-486f-8d30-e7bae812b959 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:57.978547 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b806c75-5459-4bf5-88e0-518796db98ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:57.980655 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b806c75-5459-4bf5-88e0-518796db98ee admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:57.980953 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b806c75-5459-4bf5-88e0-518796db98ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:57.981232 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b806c75-5459-4bf5-88e0-518796db98ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:57.982410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b806c75-5459-4bf5-88e0-518796db98ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:57.985332 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b806c75-5459-4bf5-88e0-518796db98ee admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:57.996110 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-5147d50c-b410-4633-8d6b-5b465dd92f07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74c33b0fcdca45888f0220d01f6870f0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:58.001179 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-5147d50c-b410-4633-8d6b-5b465dd92f07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74c33b0fcdca45888f0220d01f6870f0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:58.001179 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-5147d50c-b410-4633-8d6b-5b465dd92f07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74c33b0fcdca45888f0220d01f6870f0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:58.029410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 939/1843] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:45:58.032776 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b806c75-5459-4bf5-88e0-518796db98ee admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:58.035025 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 905/1844] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:45:57 2022] DELETE /identity/v3/users/74c33b0fcdca45888f0220d01f6870f0 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:58.039054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-5b806c75-5459-4bf5-88e0-518796db98ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:58.057111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b806c75-5459-4bf5-88e0-518796db98ee admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:58.204384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc9ac9-10bd-4443-b942-97ff39787f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:58.205379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc9ac9-10bd-4443-b942-97ff39787f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:58.205842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc9ac9-10bd-4443-b942-97ff39787f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:58.206166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc9ac9-10bd-4443-b942-97ff39787f33 admin admin] PATH_INFO: `/v3/projects/94491c37393f444880c6456030ae19c9/users/a7ed52bb1aba48f1a0454fcb7a97c0a8/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:58.213273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc9ac9-10bd-4443-b942-97ff39787f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94491c37393f444880c6456030ae19c9, user_id=a7ed52bb1aba48f1a0454fcb7a97c0a8, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:58.216193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc9ac9-10bd-4443-b942-97ff39787f33 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:58.237889 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 940/1845] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:45:58 2022] PUT /identity/v3/projects/94491c37393f444880c6456030ae19c9/users/a7ed52bb1aba48f1a0454fcb7a97c0a8/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:58.250009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb2433-a6bd-4172-be87-f11c2be9be49 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:58.250305 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb2433-a6bd-4172-be87-f11c2be9be49 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:58.250592 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb2433-a6bd-4172-be87-f11c2be9be49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:58.334736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-7feb2433-a6bd-4172-be87-f11c2be9be49 None None] MFA Rules not processed for user `a7ed52bb1aba48f1a0454fcb7a97c0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:58.336175 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-7feb2433-a6bd-4172-be87-f11c2be9be49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:58.462195 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 906/1846] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:58 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:58.471262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc9ac9-10bd-4443-b942-97ff39787f33 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:58.473032 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-57dc9ac9-10bd-4443-b942-97ff39787f33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:58.581177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc9ac9-10bd-4443-b942-97ff39787f33 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:58.586532 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bde4b6-cc0e-4d2c-9780-6b19f94e04d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:58.588364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bde4b6-cc0e-4d2c-9780-6b19f94e04d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:58.594011 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bde4b6-cc0e-4d2c-9780-6b19f94e04d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:58.594011 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bde4b6-cc0e-4d2c-9780-6b19f94e04d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:58.598930 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bde4b6-cc0e-4d2c-9780-6b19f94e04d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:58.600480 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bde4b6-cc0e-4d2c-9780-6b19f94e04d1 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:58.605633 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:58.615813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 941/1847] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:58 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:58.847777 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb2433-a6bd-4172-be87-f11c2be9be49 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:58.857897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb2433-a6bd-4172-be87-f11c2be9be49 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:58.862630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a995f-f04d-45b2-81a6-970ed2a131ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:58.862630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a995f-f04d-45b2-81a6-970ed2a131ff service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:58.862630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a995f-f04d-45b2-81a6-970ed2a131ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:58.862956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a995f-f04d-45b2-81a6-970ed2a131ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:58.864753 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3e3a995f-f04d-45b2-81a6-970ed2a131ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:58.924303 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a995f-f04d-45b2-81a6-970ed2a131ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:58.925818 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a995f-f04d-45b2-81a6-970ed2a131ff service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:58.935552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 907/1848] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:58 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:59.289806 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bde4b6-cc0e-4d2c-9780-6b19f94e04d1 service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:59.292549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a5bde4b6-cc0e-4d2c-9780-6b19f94e04d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:59.414154 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bde4b6-cc0e-4d2c-9780-6b19f94e04d1 service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:59.419278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12733cd3-8e98-4886-b491-98659571589a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:59.420180 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12733cd3-8e98-4886-b491-98659571589a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:59.420464 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12733cd3-8e98-4886-b491-98659571589a admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:59.422560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12733cd3-8e98-4886-b491-98659571589a admin admin] PATH_INFO: `/v3/projects/13f7d51259a7480fa9e90458c9f34e87` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:59.423891 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12733cd3-8e98-4886-b491-98659571589a admin admin] RBAC: Authorizing `identity:delete_project(project_id=13f7d51259a7480fa9e90458c9f34e87)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:59.425977 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12733cd3-8e98-4886-b491-98659571589a admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:59.585406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-12733cd3-8e98-4886-b491-98659571589a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13f7d51259a7480fa9e90458c9f34e87', 'request_id': 'req-12733cd3-8e98-4886-b491-98659571589a'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:59.593715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-12733cd3-8e98-4886-b491-98659571589a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13f7d51259a7480fa9e90458c9f34e87', 'request_id': 'req-12733cd3-8e98-4886-b491-98659571589a'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:59.600520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-12733cd3-8e98-4886-b491-98659571589a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13f7d51259a7480fa9e90458c9f34e87', 'request_id': 'req-12733cd3-8e98-4886-b491-98659571589a'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:59.663812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-12733cd3-8e98-4886-b491-98659571589a admin admin] The token cache is being invalidate because project 13f7d51259a7480fa9e90458c9f34e87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:45:59.664101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-12733cd3-8e98-4886-b491-98659571589a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:59.664755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-12733cd3-8e98-4886-b491-98659571589a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:45:59.666486 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 942/1849] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:45:59 2022] DELETE /identity/v3/projects/13f7d51259a7480fa9e90458c9f34e87 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:59.671609 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297eb3b-6a8a-4821-aac9-d85ca2262a97 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:59.671935 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297eb3b-6a8a-4821-aac9-d85ca2262a97 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:59.672218 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297eb3b-6a8a-4821-aac9-d85ca2262a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:59.678632 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:59.691357 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-6297eb3b-6a8a-4821-aac9-d85ca2262a97 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:59.692590 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-6297eb3b-6a8a-4821-aac9-d85ca2262a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:59.753208 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 908/1850] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:59.762012 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2702b47-26c5-495b-9ec3-417abf02a706 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:59.762334 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2702b47-26c5-495b-9ec3-417abf02a706 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:59.762616 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2702b47-26c5-495b-9ec3-417abf02a706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:59.781585 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-d2702b47-26c5-495b-9ec3-417abf02a706 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:59.782824 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d2702b47-26c5-495b-9ec3-417abf02a706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:59.792472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 943/1851] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:59.796739 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297eb3b-6a8a-4821-aac9-d85ca2262a97 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:59.806961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297eb3b-6a8a-4821-aac9-d85ca2262a97 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:59.812121 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1923b1-4802-4bf8-8c08-02792f547315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:59.812824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1923b1-4802-4bf8-8c08-02792f547315 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:59.813109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1923b1-4802-4bf8-8c08-02792f547315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:59.813386 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1923b1-4802-4bf8-8c08-02792f547315 admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:59.814120 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1923b1-4802-4bf8-8c08-02792f547315 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:45:59.814429 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1923b1-4802-4bf8-8c08-02792f547315 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:59.816033 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1923b1-4802-4bf8-8c08-02792f547315 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:59.828318 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 909/1852] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:45:59.837829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2702b47-26c5-495b-9ec3-417abf02a706 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:59.853513 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2702b47-26c5-495b-9ec3-417abf02a706 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:59.858754 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc5330-9b5d-4bf3-8371-56f675dba102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:59.861447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc5330-9b5d-4bf3-8371-56f675dba102 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:59.861447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc5330-9b5d-4bf3-8371-56f675dba102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:59.861447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc5330-9b5d-4bf3-8371-56f675dba102 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:59.862033 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc5330-9b5d-4bf3-8371-56f675dba102 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:59.864111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc5330-9b5d-4bf3-8371-56f675dba102 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:59.990577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 944/1853] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:45:59 2022] POST /identity/v3/projects => generated 375 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:45:59.991748 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1923b1-4802-4bf8-8c08-02792f547315 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:00.003866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1923b1-4802-4bf8-8c08-02792f547315 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:00.082816 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd20a3-f8d4-4a5f-bfe4-c0b110bb13a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:00.083502 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd20a3-f8d4-4a5f-bfe4-c0b110bb13a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:00.083827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd20a3-f8d4-4a5f-bfe4-c0b110bb13a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:00.087269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd20a3-f8d4-4a5f-bfe4-c0b110bb13a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:00.087269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd20a3-f8d4-4a5f-bfe4-c0b110bb13a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:00.087269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd20a3-f8d4-4a5f-bfe4-c0b110bb13a0 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:00.277064 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 910/1854] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:45:59 2022] POST /identity/v3/users => generated 288 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:00.278320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc5330-9b5d-4bf3-8371-56f675dba102 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:00.305554 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc5330-9b5d-4bf3-8371-56f675dba102 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:00.326238 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b44250-be82-41c6-9888-1c4f46cefce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:00.326939 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b44250-be82-41c6-9888-1c4f46cefce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:00.327248 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b44250-be82-41c6-9888-1c4f46cefce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:00.331313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b44250-be82-41c6-9888-1c4f46cefce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:00.331313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b44250-be82-41c6-9888-1c4f46cefce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:00.331313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b44250-be82-41c6-9888-1c4f46cefce6 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:00.345076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 945/1855] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:00.349103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd20a3-f8d4-4a5f-bfe4-c0b110bb13a0 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:00.362435 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd20a3-f8d4-4a5f-bfe4-c0b110bb13a0 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:00.367960 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03f536-9713-40ff-82a9-29aa1eaa8aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:00.368728 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03f536-9713-40ff-82a9-29aa1eaa8aeb admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:00.369185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03f536-9713-40ff-82a9-29aa1eaa8aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:00.372728 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03f536-9713-40ff-82a9-29aa1eaa8aeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:00.372728 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03f536-9713-40ff-82a9-29aa1eaa8aeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:00.372728 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03f536-9713-40ff-82a9-29aa1eaa8aeb admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:00.383323 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 911/1856] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:00.388164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b44250-be82-41c6-9888-1c4f46cefce6 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:00.402347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b44250-be82-41c6-9888-1c4f46cefce6 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:00.408524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebedfa1-a72a-4a20-bf26-4352fddd8434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:00.409735 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebedfa1-a72a-4a20-bf26-4352fddd8434 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:00.410024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebedfa1-a72a-4a20-bf26-4352fddd8434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:00.410303 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebedfa1-a72a-4a20-bf26-4352fddd8434 admin admin] PATH_INFO: `/v3/projects/6273d08f98084ffb8a005e05df29c3e0/users/cb3fdad613f447bdb1f0038f268693d4/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:00.432949 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebedfa1-a72a-4a20-bf26-4352fddd8434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6273d08f98084ffb8a005e05df29c3e0, user_id=cb3fdad613f447bdb1f0038f268693d4, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:00.435823 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebedfa1-a72a-4a20-bf26-4352fddd8434 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:00.507249 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 946/1857] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:00 2022] PUT /identity/v3/projects/6273d08f98084ffb8a005e05df29c3e0/users/cb3fdad613f447bdb1f0038f268693d4/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:00.517371 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb6ca7-7013-4baf-9170-ee6874c02a6b None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:00.517665 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb6ca7-7013-4baf-9170-ee6874c02a6b None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:00.517946 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb6ca7-7013-4baf-9170-ee6874c02a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:00.551842 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-01fb6ca7-7013-4baf-9170-ee6874c02a6b None None] MFA Rules not processed for user `cb3fdad613f447bdb1f0038f268693d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:00.553247 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-01fb6ca7-7013-4baf-9170-ee6874c02a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:00.629207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:00.639529 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 912/1858] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:00.652234 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebedfa1-a72a-4a20-bf26-4352fddd8434 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:00.654101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-5ebedfa1-a72a-4a20-bf26-4352fddd8434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:00.684152 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:00.720149 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebedfa1-a72a-4a20-bf26-4352fddd8434 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:00.723856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfecfece-b6c6-45f5-bfee-b7434a30fb9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:00.724513 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecfece-b6c6-45f5-bfee-b7434a30fb9d service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:00.724856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecfece-b6c6-45f5-bfee-b7434a30fb9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:00.725160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecfece-b6c6-45f5-bfee-b7434a30fb9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:00.734465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfecfece-b6c6-45f5-bfee-b7434a30fb9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:00.736018 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfecfece-b6c6-45f5-bfee-b7434a30fb9d service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:00.745242 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 947/1859] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:06.490613 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fe3dd-c780-42fa-8b6c-303aa3642143 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:06.490613 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fe3dd-c780-42fa-8b6c-303aa3642143 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:06.490613 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fe3dd-c780-42fa-8b6c-303aa3642143 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:06.490613 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 913/1860] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 22:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:46:06.504015 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08534f-6686-4e7c-bbd2-227eb6871eaf None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:06.504466 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08534f-6686-4e7c-bbd2-227eb6871eaf None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:06.504569 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08534f-6686-4e7c-bbd2-227eb6871eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:06.527669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-5c08534f-6686-4e7c-bbd2-227eb6871eaf None None] MFA Rules not processed for user `159cc8befc3a4d57bbd313832084db62`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:06.529072 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-5c08534f-6686-4e7c-bbd2-227eb6871eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:06.669000 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 948/1861] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 22:46:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:06.713994 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528fe3dd-c780-42fa-8b6c-303aa3642143 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:06.715799 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-528fe3dd-c780-42fa-8b6c-303aa3642143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:06.734387 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528fe3dd-c780-42fa-8b6c-303aa3642143 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:06.740700 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c701b3e-bc41-485e-87de-b4ff6d8e2ba3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:06.741553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c701b3e-bc41-485e-87de-b4ff6d8e2ba3 service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:06.741993 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c701b3e-bc41-485e-87de-b4ff6d8e2ba3 service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:06.742413 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c701b3e-bc41-485e-87de-b4ff6d8e2ba3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:06.769170 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c701b3e-bc41-485e-87de-b4ff6d8e2ba3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:06.770811 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c701b3e-bc41-485e-87de-b4ff6d8e2ba3 service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:06.782724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 914/1862] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:16.646004 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=46 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:16.777854 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=7 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:19.528823 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08534f-6686-4e7c-bbd2-227eb6871eaf None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:19.530337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-5c08534f-6686-4e7c-bbd2-227eb6871eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:19.638182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08534f-6686-4e7c-bbd2-227eb6871eaf None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:19.644152 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedb782-2e3a-4e06-852b-7281be9c6fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:19.645119 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedb782-2e3a-4e06-852b-7281be9c6fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:19.645680 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedb782-2e3a-4e06-852b-7281be9c6fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:19.645680 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedb782-2e3a-4e06-852b-7281be9c6fc7 admin admin] PATH_INFO: `/v3/users/43fd115fc9d5481eb3228327360c111f` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:19.647311 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedb782-2e3a-4e06-852b-7281be9c6fc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43fd115fc9d5481eb3228327360c111f)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:19.649061 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedb782-2e3a-4e06-852b-7281be9c6fc7 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:19.895535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6bedb782-2e3a-4e06-852b-7281be9c6fc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43fd115fc9d5481eb3228327360c111f'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:19.915584 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6bedb782-2e3a-4e06-852b-7281be9c6fc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43fd115fc9d5481eb3228327360c111f'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:19.917233 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6bedb782-2e3a-4e06-852b-7281be9c6fc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43fd115fc9d5481eb3228327360c111f'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:19.918221 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6bedb782-2e3a-4e06-852b-7281be9c6fc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43fd115fc9d5481eb3228327360c111f'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:19.928810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6bedb782-2e3a-4e06-852b-7281be9c6fc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43fd115fc9d5481eb3228327360c111f'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:20.006484 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 949/1863] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:19 2022] DELETE /identity/v3/users/43fd115fc9d5481eb3228327360c111f => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:20.011861 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147614f-a223-4bc5-84c9-858b35d062bf None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:20.012378 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147614f-a223-4bc5-84c9-858b35d062bf None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:20.012486 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147614f-a223-4bc5-84c9-858b35d062bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:20.032998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-b147614f-a223-4bc5-84c9-858b35d062bf None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:20.034295 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-b147614f-a223-4bc5-84c9-858b35d062bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:20.097974 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 915/1864] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:46:20.108156 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedb782-2e3a-4e06-852b-7281be9c6fc7 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:20.109824 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6bedb782-2e3a-4e06-852b-7281be9c6fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:20.169702 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedb782-2e3a-4e06-852b-7281be9c6fc7 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:20.173323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a9cc22-9db1-4e1c-9ffa-c1c1101f6185 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:20.176887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9cc22-9db1-4e1c-9ffa-c1c1101f6185 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:20.176887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9cc22-9db1-4e1c-9ffa-c1c1101f6185 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:20.176887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9cc22-9db1-4e1c-9ffa-c1c1101f6185 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:20.185526 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a9cc22-9db1-4e1c-9ffa-c1c1101f6185 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:20.187002 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a9cc22-9db1-4e1c-9ffa-c1c1101f6185 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:20.197336 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 950/1865] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:20.917953 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147614f-a223-4bc5-84c9-858b35d062bf None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:20.919482 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-b147614f-a223-4bc5-84c9-858b35d062bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:20.936910 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147614f-a223-4bc5-84c9-858b35d062bf None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:20.942363 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de51be-62bc-4233-93c3-a0308325c406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:20.943534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de51be-62bc-4233-93c3-a0308325c406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:20.944280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de51be-62bc-4233-93c3-a0308325c406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:20.944803 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de51be-62bc-4233-93c3-a0308325c406 admin admin] PATH_INFO: `/v3/projects/3711628a625146f388cb276e025da7de` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:20.946329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de51be-62bc-4233-93c3-a0308325c406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3711628a625146f388cb276e025da7de)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:20.949626 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de51be-62bc-4233-93c3-a0308325c406 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:21.119863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-06de51be-62bc-4233-93c3-a0308325c406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3711628a625146f388cb276e025da7de', 'request_id': 'req-06de51be-62bc-4233-93c3-a0308325c406'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:21.167130 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-06de51be-62bc-4233-93c3-a0308325c406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3711628a625146f388cb276e025da7de', 'request_id': 'req-06de51be-62bc-4233-93c3-a0308325c406'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:21.174669 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-06de51be-62bc-4233-93c3-a0308325c406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3711628a625146f388cb276e025da7de', 'request_id': 'req-06de51be-62bc-4233-93c3-a0308325c406'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:21.187188 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-06de51be-62bc-4233-93c3-a0308325c406 admin admin] The token cache is being invalidate because project 3711628a625146f388cb276e025da7de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:21.194108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-06de51be-62bc-4233-93c3-a0308325c406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:21.195263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-06de51be-62bc-4233-93c3-a0308325c406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:21.208332 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 916/1866] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:20 2022] DELETE /identity/v3/projects/3711628a625146f388cb276e025da7de => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:21.209195 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a9cc22-9db1-4e1c-9ffa-c1c1101f6185 service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:21.224089 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a9cc22-9db1-4e1c-9ffa-c1c1101f6185 service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:21.250590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:21.295317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcbd5c-0465-45f6-95ad-c5e9bfc780dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:21.296401 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcbd5c-0465-45f6-95ad-c5e9bfc780dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:21.296719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcbd5c-0465-45f6-95ad-c5e9bfc780dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:21.297005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcbd5c-0465-45f6-95ad-c5e9bfc780dd admin admin] PATH_INFO: `/v3/users/80ed460262494822a370a5b70b5b9383` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:21.298209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbcbd5c-0465-45f6-95ad-c5e9bfc780dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=80ed460262494822a370a5b70b5b9383)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:21.300124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbcbd5c-0465-45f6-95ad-c5e9bfc780dd admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:21.478478 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-0dbcbd5c-0465-45f6-95ad-c5e9bfc780dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80ed460262494822a370a5b70b5b9383'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:21.502169 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-0dbcbd5c-0465-45f6-95ad-c5e9bfc780dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80ed460262494822a370a5b70b5b9383'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:21.503053 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-0dbcbd5c-0465-45f6-95ad-c5e9bfc780dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80ed460262494822a370a5b70b5b9383'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:21.504343 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-0dbcbd5c-0465-45f6-95ad-c5e9bfc780dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80ed460262494822a370a5b70b5b9383'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:21.513566 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-0dbcbd5c-0465-45f6-95ad-c5e9bfc780dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80ed460262494822a370a5b70b5b9383'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:21.620965 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 951/1867] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:21 2022] DELETE /identity/v3/users/80ed460262494822a370a5b70b5b9383 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:22.303651 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de51be-62bc-4233-93c3-a0308325c406 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:22.305277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-06de51be-62bc-4233-93c3-a0308325c406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:22.319443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de51be-62bc-4233-93c3-a0308325c406 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:22.391595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8961a4e-f462-4e10-9f38-134dbbd395c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:22.392490 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8961a4e-f462-4e10-9f38-134dbbd395c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:22.392836 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8961a4e-f462-4e10-9f38-134dbbd395c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:22.393522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8961a4e-f462-4e10-9f38-134dbbd395c8 admin admin] PATH_INFO: `/v3/users/cb3fdad613f447bdb1f0038f268693d4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:22.394490 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8961a4e-f462-4e10-9f38-134dbbd395c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb3fdad613f447bdb1f0038f268693d4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:22.396331 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8961a4e-f462-4e10-9f38-134dbbd395c8 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:22.462057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcbd5c-0465-45f6-95ad-c5e9bfc780dd admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:22.466092 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-0dbcbd5c-0465-45f6-95ad-c5e9bfc780dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:22.483142 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcbd5c-0465-45f6-95ad-c5e9bfc780dd admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:22.488712 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a2b83-1e3f-476f-993e-0f2c8355af08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:22.490415 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a2b83-1e3f-476f-993e-0f2c8355af08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:22.490747 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a2b83-1e3f-476f-993e-0f2c8355af08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:22.491033 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a2b83-1e3f-476f-993e-0f2c8355af08 admin admin] PATH_INFO: `/v3/projects/7f2eb7f2ca5c45c3bb9aedcdf6362fff` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:22.492395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a2b83-1e3f-476f-993e-0f2c8355af08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f2eb7f2ca5c45c3bb9aedcdf6362fff)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:22.495217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a2b83-1e3f-476f-993e-0f2c8355af08 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:22.516410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:22.623518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-c8961a4e-f462-4e10-9f38-134dbbd395c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb3fdad613f447bdb1f0038f268693d4'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:22.644818 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-c8961a4e-f462-4e10-9f38-134dbbd395c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb3fdad613f447bdb1f0038f268693d4'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:22.670251 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-843a2b83-1e3f-476f-993e-0f2c8355af08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f2eb7f2ca5c45c3bb9aedcdf6362fff', 'request_id': 'req-843a2b83-1e3f-476f-993e-0f2c8355af08'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:22.677467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-843a2b83-1e3f-476f-993e-0f2c8355af08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f2eb7f2ca5c45c3bb9aedcdf6362fff', 'request_id': 'req-843a2b83-1e3f-476f-993e-0f2c8355af08'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:22.685782 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-843a2b83-1e3f-476f-993e-0f2c8355af08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f2eb7f2ca5c45c3bb9aedcdf6362fff', 'request_id': 'req-843a2b83-1e3f-476f-993e-0f2c8355af08'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:22.727587 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-c8961a4e-f462-4e10-9f38-134dbbd395c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb3fdad613f447bdb1f0038f268693d4'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:22.729408 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-c8961a4e-f462-4e10-9f38-134dbbd395c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb3fdad613f447bdb1f0038f268693d4'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:22.731354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-c8961a4e-f462-4e10-9f38-134dbbd395c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb3fdad613f447bdb1f0038f268693d4'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:22.773690 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 917/1868] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:22 2022] DELETE /identity/v3/users/cb3fdad613f447bdb1f0038f268693d4 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:22.785119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfe9c8-6ad1-43e2-9bb1-c49a1cab68a7 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:22.786480 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfe9c8-6ad1-43e2-9bb1-c49a1cab68a7 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:22.787123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfe9c8-6ad1-43e2-9bb1-c49a1cab68a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:22.813970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-f5cfe9c8-6ad1-43e2-9bb1-c49a1cab68a7 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:22.813970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-f5cfe9c8-6ad1-43e2-9bb1-c49a1cab68a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:22.837602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-843a2b83-1e3f-476f-993e-0f2c8355af08 admin admin] The token cache is being invalidate because project 7f2eb7f2ca5c45c3bb9aedcdf6362fff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:22.842708 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-843a2b83-1e3f-476f-993e-0f2c8355af08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:22.842708 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-843a2b83-1e3f-476f-993e-0f2c8355af08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:22.844062 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 952/1869] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:22 2022] DELETE /identity/v3/projects/7f2eb7f2ca5c45c3bb9aedcdf6362fff => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:46:22.847721 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66724837-7408-4478-81a8-2814ee44748e None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:22.850734 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66724837-7408-4478-81a8-2814ee44748e None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:22.851284 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66724837-7408-4478-81a8-2814ee44748e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:22.881602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-66724837-7408-4478-81a8-2814ee44748e None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:22.883672 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-66724837-7408-4478-81a8-2814ee44748e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:22.887685 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 918/1870] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:22.895950 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 953/1871] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:22.898017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfe9c8-6ad1-43e2-9bb1-c49a1cab68a7 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:22.899801 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-f5cfe9c8-6ad1-43e2-9bb1-c49a1cab68a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:22.903881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96dac3-972f-487c-93de-1232553a0e95 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:22.906207 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96dac3-972f-487c-93de-1232553a0e95 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:22.906493 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96dac3-972f-487c-93de-1232553a0e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:22.948276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-0f96dac3-972f-487c-93de-1232553a0e95 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:22.950780 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-0f96dac3-972f-487c-93de-1232553a0e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:22.969495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 954/1872] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:22.973093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfe9c8-6ad1-43e2-9bb1-c49a1cab68a7 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:22.973240 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96dac3-972f-487c-93de-1232553a0e95 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:22.974285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48583da-a747-46f6-9f24-a972fc877bc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:22.974994 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48583da-a747-46f6-9f24-a972fc877bc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:22.975285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48583da-a747-46f6-9f24-a972fc877bc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:22.975582 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48583da-a747-46f6-9f24-a972fc877bc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:22.981719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96dac3-972f-487c-93de-1232553a0e95 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:22.986531 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48583da-a747-46f6-9f24-a972fc877bc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:22.986683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67c987-f44d-41e2-8f70-26abf2e271b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:22.987241 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67c987-f44d-41e2-8f70-26abf2e271b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:22.987531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67c987-f44d-41e2-8f70-26abf2e271b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:22.987857 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67c987-f44d-41e2-8f70-26abf2e271b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:22.988165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48583da-a747-46f6-9f24-a972fc877bc4 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:22.988546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67c987-f44d-41e2-8f70-26abf2e271b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:46:22.988886 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67c987-f44d-41e2-8f70-26abf2e271b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:22.990641 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67c987-f44d-41e2-8f70-26abf2e271b6 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:23.000349 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:23.004062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 919/1873] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:46:23.012972 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 955/1874] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:46:23.016303 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48583da-a747-46f6-9f24-a972fc877bc4 service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:23.027435 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48583da-a747-46f6-9f24-a972fc877bc4 service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:23.032360 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ad24c-73ed-465b-9a79-121eddcc510f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:23.033196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ad24c-73ed-465b-9a79-121eddcc510f admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:23.033598 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ad24c-73ed-465b-9a79-121eddcc510f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:23.033881 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ad24c-73ed-465b-9a79-121eddcc510f admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:23.034959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ad24c-73ed-465b-9a79-121eddcc510f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:23.036675 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ad24c-73ed-465b-9a79-121eddcc510f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:23.140900 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 920/1875] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:23 2022] POST /identity/v3/projects => generated 367 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:23.145394 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67c987-f44d-41e2-8f70-26abf2e271b6 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:23.156094 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67c987-f44d-41e2-8f70-26abf2e271b6 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:23.230482 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee0cda-90f7-4ebf-88be-286686375125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:23.231166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee0cda-90f7-4ebf-88be-286686375125 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:23.231451 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee0cda-90f7-4ebf-88be-286686375125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:23.231764 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee0cda-90f7-4ebf-88be-286686375125 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:23.232458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee0cda-90f7-4ebf-88be-286686375125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:23.234476 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee0cda-90f7-4ebf-88be-286686375125 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:23.309011 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 956/1876] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:23 2022] POST /identity/v3/users => generated 284 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:23.310387 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ad24c-73ed-465b-9a79-121eddcc510f admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:23.326197 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ad24c-73ed-465b-9a79-121eddcc510f admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:23.339480 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb90fb7-210c-4c80-ab0f-c59686bc458e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:23.341036 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb90fb7-210c-4c80-ab0f-c59686bc458e admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:23.341635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb90fb7-210c-4c80-ab0f-c59686bc458e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:23.342203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb90fb7-210c-4c80-ab0f-c59686bc458e admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:23.343573 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb90fb7-210c-4c80-ab0f-c59686bc458e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:23.346224 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb90fb7-210c-4c80-ab0f-c59686bc458e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:23.364317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 921/1877] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:23.364797 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee0cda-90f7-4ebf-88be-286686375125 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:23.397099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee0cda-90f7-4ebf-88be-286686375125 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:23.418198 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a35c13-14d3-4ac0-905f-8c955aa0ba0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:23.422086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a35c13-14d3-4ac0-905f-8c955aa0ba0b admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:23.422380 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a35c13-14d3-4ac0-905f-8c955aa0ba0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:23.422812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a35c13-14d3-4ac0-905f-8c955aa0ba0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:23.424529 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a35c13-14d3-4ac0-905f-8c955aa0ba0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:23.427103 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a35c13-14d3-4ac0-905f-8c955aa0ba0b admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:23.453378 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 957/1878] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:23.457711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb90fb7-210c-4c80-ab0f-c59686bc458e admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:23.506415 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb90fb7-210c-4c80-ab0f-c59686bc458e admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:23.530623 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f982a7a-9ea6-4088-aefb-ac0c77de9efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:23.539279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f982a7a-9ea6-4088-aefb-ac0c77de9efb admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:23.539644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f982a7a-9ea6-4088-aefb-ac0c77de9efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:23.539938 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f982a7a-9ea6-4088-aefb-ac0c77de9efb admin admin] PATH_INFO: `/v3/projects/933706baf68c42f69bd41249d5c0faef/users/56b27cbf1e7149bd81a60bcb1e504670/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:23.568117 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:23.576193 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f982a7a-9ea6-4088-aefb-ac0c77de9efb admin admin] RBAC: Authorizing `identity:create_grant(project_id=933706baf68c42f69bd41249d5c0faef, user_id=56b27cbf1e7149bd81a60bcb1e504670, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:23.587914 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f982a7a-9ea6-4088-aefb-ac0c77de9efb admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:23.691056 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 922/1879] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:23 2022] PUT /identity/v3/projects/933706baf68c42f69bd41249d5c0faef/users/56b27cbf1e7149bd81a60bcb1e504670/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:23.708557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a731ef-919e-4caf-beac-185e7308e081 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:23.709577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a731ef-919e-4caf-beac-185e7308e081 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:23.709900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a731ef-919e-4caf-beac-185e7308e081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:23.769530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-b7a731ef-919e-4caf-beac-185e7308e081 None None] MFA Rules not processed for user `56b27cbf1e7149bd81a60bcb1e504670`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:23.770906 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-b7a731ef-919e-4caf-beac-185e7308e081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:23.886817 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 958/1880] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:23 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:23.891421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f982a7a-9ea6-4088-aefb-ac0c77de9efb admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:23.986349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f982a7a-9ea6-4088-aefb-ac0c77de9efb admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:23.992056 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b071ade-1e3c-41f1-a033-c42f792e453e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:23.992776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b071ade-1e3c-41f1-a033-c42f792e453e admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:23.993066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b071ade-1e3c-41f1-a033-c42f792e453e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:23.993592 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b071ade-1e3c-41f1-a033-c42f792e453e admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:23.994618 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b071ade-1e3c-41f1-a033-c42f792e453e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:23.996859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b071ade-1e3c-41f1-a033-c42f792e453e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:24.141822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a731ef-919e-4caf-beac-185e7308e081 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:24.153638 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 923/1881] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:23 2022] POST /identity/v3/projects => generated 367 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:24.156611 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a731ef-919e-4caf-beac-185e7308e081 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:24.220809 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b071ade-1e3c-41f1-a033-c42f792e453e admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:24.222543 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-0b071ade-1e3c-41f1-a033-c42f792e453e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:24.251621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbc5e0-5a80-4361-9892-151a6595599a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:24.253095 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbc5e0-5a80-4361-9892-151a6595599a admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:24.253657 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbc5e0-5a80-4361-9892-151a6595599a admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:24.253945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbc5e0-5a80-4361-9892-151a6595599a admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:24.254650 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccbc5e0-5a80-4361-9892-151a6595599a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:24.261374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccbc5e0-5a80-4361-9892-151a6595599a admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:24.287727 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:24.339642 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b071ade-1e3c-41f1-a033-c42f792e453e admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:24.345793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6c81a-d588-410e-a7a0-f8d894b223c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:24.346673 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6c81a-d588-410e-a7a0-f8d894b223c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:24.346958 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6c81a-d588-410e-a7a0-f8d894b223c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:24.347235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6c81a-d588-410e-a7a0-f8d894b223c2 admin admin] PATH_INFO: `/v3/projects/6273d08f98084ffb8a005e05df29c3e0` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:24.348427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6c81a-d588-410e-a7a0-f8d894b223c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6273d08f98084ffb8a005e05df29c3e0)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:24.350471 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6c81a-d588-410e-a7a0-f8d894b223c2 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:24.373306 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 959/1882] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:24 2022] POST /identity/v3/users => generated 290 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:24.379103 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbc5e0-5a80-4361-9892-151a6595599a admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:24.393895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbc5e0-5a80-4361-9892-151a6595599a admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:24.399888 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e63fb-25dc-49fd-add2-5bd5d581ad51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:24.401354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e63fb-25dc-49fd-add2-5bd5d581ad51 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:24.402285 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e63fb-25dc-49fd-add2-5bd5d581ad51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:24.403286 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e63fb-25dc-49fd-add2-5bd5d581ad51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:24.403659 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e63fb-25dc-49fd-add2-5bd5d581ad51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:24.405149 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e63fb-25dc-49fd-add2-5bd5d581ad51 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:24.428359 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 960/1883] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:24.433191 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e63fb-25dc-49fd-add2-5bd5d581ad51 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:24.444462 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e63fb-25dc-49fd-add2-5bd5d581ad51 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:24.450256 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7cc36-63a9-4fed-a4aa-8ac63213f436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:24.455696 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7cc36-63a9-4fed-a4aa-8ac63213f436 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:24.459077 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7cc36-63a9-4fed-a4aa-8ac63213f436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:24.459077 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7cc36-63a9-4fed-a4aa-8ac63213f436 admin admin] PATH_INFO: `/v3/projects/a96b14fe70754624806e9252d5ebc21c/users/4ac94628e02c4e0aaa02149ff9707229/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:24.481379 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-22d6c81a-d588-410e-a7a0-f8d894b223c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6273d08f98084ffb8a005e05df29c3e0', 'request_id': 'req-22d6c81a-d588-410e-a7a0-f8d894b223c2'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:24.484071 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7cc36-63a9-4fed-a4aa-8ac63213f436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a96b14fe70754624806e9252d5ebc21c, user_id=4ac94628e02c4e0aaa02149ff9707229, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:24.486624 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7cc36-63a9-4fed-a4aa-8ac63213f436 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:24.555445 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-22d6c81a-d588-410e-a7a0-f8d894b223c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6273d08f98084ffb8a005e05df29c3e0', 'request_id': 'req-22d6c81a-d588-410e-a7a0-f8d894b223c2'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:24.568491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-22d6c81a-d588-410e-a7a0-f8d894b223c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6273d08f98084ffb8a005e05df29c3e0', 'request_id': 'req-22d6c81a-d588-410e-a7a0-f8d894b223c2'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:24.580093 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 961/1884] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:24 2022] PUT /identity/v3/projects/a96b14fe70754624806e9252d5ebc21c/users/4ac94628e02c4e0aaa02149ff9707229/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:46:24.583059 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-22d6c81a-d588-410e-a7a0-f8d894b223c2 admin admin] The token cache is being invalidate because project 6273d08f98084ffb8a005e05df29c3e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:24.583929 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-22d6c81a-d588-410e-a7a0-f8d894b223c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:24.584438 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7cc36-63a9-4fed-a4aa-8ac63213f436 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:24.584783 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-22d6c81a-d588-410e-a7a0-f8d894b223c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:24.586497 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-f9a7cc36-63a9-4fed-a4aa-8ac63213f436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:24.586696 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 924/1885] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:24 2022] DELETE /identity/v3/projects/6273d08f98084ffb8a005e05df29c3e0 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:46:24.593726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec4d5e-6e3a-475a-ac0d-72a9b5926440 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:24.594150 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec4d5e-6e3a-475a-ac0d-72a9b5926440 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:24.594642 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec4d5e-6e3a-475a-ac0d-72a9b5926440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:24.610939 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:24.628072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-77ec4d5e-6e3a-475a-ac0d-72a9b5926440 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:24.630103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-77ec4d5e-6e3a-475a-ac0d-72a9b5926440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:24.690227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7cc36-63a9-4fed-a4aa-8ac63213f436 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:24.695769 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc49efe-133d-43d6-ac59-cd5e65caaa66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:24.696595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc49efe-133d-43d6-ac59-cd5e65caaa66 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:24.697109 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc49efe-133d-43d6-ac59-cd5e65caaa66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:24.697450 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc49efe-133d-43d6-ac59-cd5e65caaa66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:24.698265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc49efe-133d-43d6-ac59-cd5e65caaa66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:24.699931 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc49efe-133d-43d6-ac59-cd5e65caaa66 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:24.711979 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 925/1886] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:24.715476 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 962/1887] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:46:24.721474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec4d5e-6e3a-475a-ac0d-72a9b5926440 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:24.722278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747aabe-ce24-4845-985f-03b1037bd060 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:24.722574 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747aabe-ce24-4845-985f-03b1037bd060 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:24.723048 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747aabe-ce24-4845-985f-03b1037bd060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:24.734348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec4d5e-6e3a-475a-ac0d-72a9b5926440 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:24.741381 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cfb62-24fa-4cad-b210-5a4b5dffeb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:24.742517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cfb62-24fa-4cad-b210-5a4b5dffeb6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:24.744281 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cfb62-24fa-4cad-b210-5a4b5dffeb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:24.744281 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cfb62-24fa-4cad-b210-5a4b5dffeb6b admin admin] PATH_INFO: `/v3/projects/a96b14fe70754624806e9252d5ebc21c/users/4ac94628e02c4e0aaa02149ff9707229/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:24.746556 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cfb62-24fa-4cad-b210-5a4b5dffeb6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a96b14fe70754624806e9252d5ebc21c, user_id=4ac94628e02c4e0aaa02149ff9707229, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:24.749026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-6747aabe-ce24-4845-985f-03b1037bd060 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:24.749973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cfb62-24fa-4cad-b210-5a4b5dffeb6b admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:24.750851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6747aabe-ce24-4845-985f-03b1037bd060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:24.761420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 963/1888] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:46:24.764774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 926/1889] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:24 2022] PUT /identity/v3/projects/a96b14fe70754624806e9252d5ebc21c/users/4ac94628e02c4e0aaa02149ff9707229/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:24.766461 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747aabe-ce24-4845-985f-03b1037bd060 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:24.777443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05defcf-3c72-4b43-b07c-7152c67318c7 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:24.777748 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05defcf-3c72-4b43-b07c-7152c67318c7 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:24.778128 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05defcf-3c72-4b43-b07c-7152c67318c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:24.779870 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747aabe-ce24-4845-985f-03b1037bd060 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:24.810625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-c05defcf-3c72-4b43-b07c-7152c67318c7 None None] MFA Rules not processed for user `4ac94628e02c4e0aaa02149ff9707229`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:24.812149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c05defcf-3c72-4b43-b07c-7152c67318c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:24.869263 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960ca54-e24c-4aca-bd5c-01f5da7dfadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:24.870175 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960ca54-e24c-4aca-bd5c-01f5da7dfadc admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:24.870554 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960ca54-e24c-4aca-bd5c-01f5da7dfadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:24.872216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960ca54-e24c-4aca-bd5c-01f5da7dfadc admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:24.872216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960ca54-e24c-4aca-bd5c-01f5da7dfadc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:46:24.872483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960ca54-e24c-4aca-bd5c-01f5da7dfadc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:24.874353 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960ca54-e24c-4aca-bd5c-01f5da7dfadc admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:24.894389 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 964/1890] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:46:24.898785 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960ca54-e24c-4aca-bd5c-01f5da7dfadc admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:24.910676 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960ca54-e24c-4aca-bd5c-01f5da7dfadc admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:24.916348 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89656b7f-c413-45f9-b69e-22da93018d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:24.917153 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89656b7f-c413-45f9-b69e-22da93018d33 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:24.917443 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89656b7f-c413-45f9-b69e-22da93018d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:24.917723 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89656b7f-c413-45f9-b69e-22da93018d33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:24.918487 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89656b7f-c413-45f9-b69e-22da93018d33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:24.920350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89656b7f-c413-45f9-b69e-22da93018d33 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:24.923087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 927/1891] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:24 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:24.936173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05defcf-3c72-4b43-b07c-7152c67318c7 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:24.939158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c05defcf-3c72-4b43-b07c-7152c67318c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:25.003726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05defcf-3c72-4b43-b07c-7152c67318c7 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:25.008113 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65a628-17f1-487d-af77-2f02c526e869 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:25.008856 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65a628-17f1-487d-af77-2f02c526e869 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:25.009184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65a628-17f1-487d-af77-2f02c526e869 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:25.009641 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65a628-17f1-487d-af77-2f02c526e869 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:25.022784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65a628-17f1-487d-af77-2f02c526e869 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:25.024251 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65a628-17f1-487d-af77-2f02c526e869 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:25.024649 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 965/1892] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:24 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:46:25.025457 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89656b7f-c413-45f9-b69e-22da93018d33 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:25.037532 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 928/1893] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:24 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:46:25.055163 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:25.097490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89656b7f-c413-45f9-b69e-22da93018d33 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:25.102733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d7bfc-f125-4145-95b9-ab131f0f25bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:25.104282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d7bfc-f125-4145-95b9-ab131f0f25bf admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:25.104786 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d7bfc-f125-4145-95b9-ab131f0f25bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:25.105177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d7bfc-f125-4145-95b9-ab131f0f25bf admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:25.106061 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d7bfc-f125-4145-95b9-ab131f0f25bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:25.108238 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d7bfc-f125-4145-95b9-ab131f0f25bf admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:25.185672 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65a628-17f1-487d-af77-2f02c526e869 service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:25.196627 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65a628-17f1-487d-af77-2f02c526e869 service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:25.210560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 966/1894] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:25 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:25.216010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d7bfc-f125-4145-95b9-ab131f0f25bf admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:25.230440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d7bfc-f125-4145-95b9-ab131f0f25bf admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:25.238209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0e015-4a81-451e-8d95-aaeac4675fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:25.240849 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0e015-4a81-451e-8d95-aaeac4675fea admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:25.241379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0e015-4a81-451e-8d95-aaeac4675fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:25.244801 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0e015-4a81-451e-8d95-aaeac4675fea admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:25.245799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0e015-4a81-451e-8d95-aaeac4675fea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:25.247841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0e015-4a81-451e-8d95-aaeac4675fea admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:25.250084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:25.283602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 967/1895] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:46:25.287587 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0e015-4a81-451e-8d95-aaeac4675fea admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:25.301779 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0e015-4a81-451e-8d95-aaeac4675fea admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:25.308922 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff8f80-655d-426c-857f-a8ce68c5238a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:25.309600 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff8f80-655d-426c-857f-a8ce68c5238a service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:25.309893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff8f80-655d-426c-857f-a8ce68c5238a service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:25.310176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff8f80-655d-426c-857f-a8ce68c5238a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:25.311265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9e455-38f5-4bfe-95e0-013447ee7061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:25.312754 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9e455-38f5-4bfe-95e0-013447ee7061 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:25.312966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3dff8f80-655d-426c-857f-a8ce68c5238a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:25.317577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9e455-38f5-4bfe-95e0-013447ee7061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:25.322134 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9e455-38f5-4bfe-95e0-013447ee7061 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:25.323827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9e455-38f5-4bfe-95e0-013447ee7061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:25.326586 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9e455-38f5-4bfe-95e0-013447ee7061 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:25.351589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 968/1896] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:25.353055 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9e455-38f5-4bfe-95e0-013447ee7061 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:25.369990 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9e455-38f5-4bfe-95e0-013447ee7061 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:25.379889 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db86dc5-a932-4552-b5d9-c04c868c9ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:25.380871 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db86dc5-a932-4552-b5d9-c04c868c9ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:25.381437 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db86dc5-a932-4552-b5d9-c04c868c9ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:25.381611 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db86dc5-a932-4552-b5d9-c04c868c9ea7 admin admin] PATH_INFO: `/v3/projects/6fdc472916934fac985e88fb6a61290b/users/84a5fb262284400591cf29dd13771ee7/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:25.399602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db86dc5-a932-4552-b5d9-c04c868c9ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fdc472916934fac985e88fb6a61290b, user_id=84a5fb262284400591cf29dd13771ee7, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:25.399793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff8f80-655d-426c-857f-a8ce68c5238a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:25.401877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff8f80-655d-426c-857f-a8ce68c5238a service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:25.405489 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db86dc5-a932-4552-b5d9-c04c868c9ea7 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:25.412495 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 929/1897] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:25 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:46:25.512910 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 969/1898] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:25 2022] PUT /identity/v3/projects/6fdc472916934fac985e88fb6a61290b/users/84a5fb262284400591cf29dd13771ee7/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:25.517949 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0bab6-b623-4af0-9f9c-dd6af4b17b49 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:25.517949 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0bab6-b623-4af0-9f9c-dd6af4b17b49 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:25.517949 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0bab6-b623-4af0-9f9c-dd6af4b17b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:25.540003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-23c0bab6-b623-4af0-9f9c-dd6af4b17b49 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:25.542088 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-23c0bab6-b623-4af0-9f9c-dd6af4b17b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:25.610804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 930/1899] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:25.622668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db86dc5-a932-4552-b5d9-c04c868c9ea7 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:25.624562 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3db86dc5-a932-4552-b5d9-c04c868c9ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:25.693101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db86dc5-a932-4552-b5d9-c04c868c9ea7 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:25.697673 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87b825-ceb5-4a7c-bee7-e26fc2e9b0b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:25.698402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87b825-ceb5-4a7c-bee7-e26fc2e9b0b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:25.698738 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87b825-ceb5-4a7c-bee7-e26fc2e9b0b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:25.699027 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87b825-ceb5-4a7c-bee7-e26fc2e9b0b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:25.710200 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87b825-ceb5-4a7c-bee7-e26fc2e9b0b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:25.710963 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87b825-ceb5-4a7c-bee7-e26fc2e9b0b0 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:25.722502 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 970/1900] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:27.070671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0bab6-b623-4af0-9f9c-dd6af4b17b49 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:27.072213 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-23c0bab6-b623-4af0-9f9c-dd6af4b17b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:27.084928 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0bab6-b623-4af0-9f9c-dd6af4b17b49 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:27.089990 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be15ee-747f-4cc9-b5b1-93a5adb3f289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:27.090855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be15ee-747f-4cc9-b5b1-93a5adb3f289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:27.091139 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be15ee-747f-4cc9-b5b1-93a5adb3f289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:27.091415 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be15ee-747f-4cc9-b5b1-93a5adb3f289 admin admin] PATH_INFO: `/v3/users/fbdbbeecfe52453cae35029faa599153` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:27.092628 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be15ee-747f-4cc9-b5b1-93a5adb3f289 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbdbbeecfe52453cae35029faa599153)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:27.094716 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be15ee-747f-4cc9-b5b1-93a5adb3f289 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:27.394831 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-39be15ee-747f-4cc9-b5b1-93a5adb3f289 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbdbbeecfe52453cae35029faa599153'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:27.443723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-39be15ee-747f-4cc9-b5b1-93a5adb3f289 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbdbbeecfe52453cae35029faa599153'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:27.454840 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:27.538143 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-39be15ee-747f-4cc9-b5b1-93a5adb3f289 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbdbbeecfe52453cae35029faa599153'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:27.539204 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-39be15ee-747f-4cc9-b5b1-93a5adb3f289 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbdbbeecfe52453cae35029faa599153'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:27.540076 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-39be15ee-747f-4cc9-b5b1-93a5adb3f289 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbdbbeecfe52453cae35029faa599153'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:27.561379 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 931/1901] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:27 2022] DELETE /identity/v3/users/fbdbbeecfe52453cae35029faa599153 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:27.565919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b449b-764e-41cc-8522-7b0ca3894ddf None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:27.566396 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b449b-764e-41cc-8522-7b0ca3894ddf None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:27.566636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b449b-764e-41cc-8522-7b0ca3894ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:27.600899 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-c30b449b-764e-41cc-8522-7b0ca3894ddf None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:27.602142 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-c30b449b-764e-41cc-8522-7b0ca3894ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:27.677084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 971/1902] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:46:27.685373 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be15ee-747f-4cc9-b5b1-93a5adb3f289 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:27.687203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-39be15ee-747f-4cc9-b5b1-93a5adb3f289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:27.786733 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be15ee-747f-4cc9-b5b1-93a5adb3f289 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:27.794357 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554ec60-7f57-4095-8b34-a841ce2ff605 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:27.795858 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554ec60-7f57-4095-8b34-a841ce2ff605 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:27.797623 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554ec60-7f57-4095-8b34-a841ce2ff605 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:27.798053 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554ec60-7f57-4095-8b34-a841ce2ff605 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:27.821421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4554ec60-7f57-4095-8b34-a841ce2ff605 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:27.823583 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4554ec60-7f57-4095-8b34-a841ce2ff605 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:27.863065 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 932/1903] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:28.527912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b449b-764e-41cc-8522-7b0ca3894ddf None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:28.530998 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-c30b449b-764e-41cc-8522-7b0ca3894ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:28.550892 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b449b-764e-41cc-8522-7b0ca3894ddf None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:28.563796 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a701917-565e-4dd8-8c22-7b51de27627f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:28.566862 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a701917-565e-4dd8-8c22-7b51de27627f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:28.566862 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a701917-565e-4dd8-8c22-7b51de27627f admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:28.566862 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a701917-565e-4dd8-8c22-7b51de27627f admin admin] PATH_INFO: `/v3/projects/b8a97d33db024a47b8c9862e53d14664` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:28.568545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a701917-565e-4dd8-8c22-7b51de27627f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8a97d33db024a47b8c9862e53d14664)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:28.571879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a701917-565e-4dd8-8c22-7b51de27627f admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:28.576020 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:28.706703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3a701917-565e-4dd8-8c22-7b51de27627f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8a97d33db024a47b8c9862e53d14664', 'request_id': 'req-3a701917-565e-4dd8-8c22-7b51de27627f'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:28.728450 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3a701917-565e-4dd8-8c22-7b51de27627f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8a97d33db024a47b8c9862e53d14664', 'request_id': 'req-3a701917-565e-4dd8-8c22-7b51de27627f'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:28.743079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3a701917-565e-4dd8-8c22-7b51de27627f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8a97d33db024a47b8c9862e53d14664', 'request_id': 'req-3a701917-565e-4dd8-8c22-7b51de27627f'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:28.808011 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3a701917-565e-4dd8-8c22-7b51de27627f admin admin] The token cache is being invalidate because project b8a97d33db024a47b8c9862e53d14664 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:28.808764 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3a701917-565e-4dd8-8c22-7b51de27627f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:28.810275 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3a701917-565e-4dd8-8c22-7b51de27627f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:28.812248 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 972/1904] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:28 2022] DELETE /identity/v3/projects/b8a97d33db024a47b8c9862e53d14664 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:28.823124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554ec60-7f57-4095-8b34-a841ce2ff605 service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:28.846013 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554ec60-7f57-4095-8b34-a841ce2ff605 service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:28.934460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d1efb-cad8-49c6-9be4-2d2e779eb390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:28.935473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d1efb-cad8-49c6-9be4-2d2e779eb390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:28.936083 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d1efb-cad8-49c6-9be4-2d2e779eb390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:28.936629 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d1efb-cad8-49c6-9be4-2d2e779eb390 admin admin] PATH_INFO: `/v3/users/a7ed52bb1aba48f1a0454fcb7a97c0a8` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:28.938175 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d1efb-cad8-49c6-9be4-2d2e779eb390 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7ed52bb1aba48f1a0454fcb7a97c0a8)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:28.940324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d1efb-cad8-49c6-9be4-2d2e779eb390 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:29.130481 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-578d1efb-cad8-49c6-9be4-2d2e779eb390 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7ed52bb1aba48f1a0454fcb7a97c0a8'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:29.145871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-578d1efb-cad8-49c6-9be4-2d2e779eb390 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7ed52bb1aba48f1a0454fcb7a97c0a8'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:29.212072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-578d1efb-cad8-49c6-9be4-2d2e779eb390 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7ed52bb1aba48f1a0454fcb7a97c0a8'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:29.213056 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-578d1efb-cad8-49c6-9be4-2d2e779eb390 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7ed52bb1aba48f1a0454fcb7a97c0a8'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:29.213761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-578d1efb-cad8-49c6-9be4-2d2e779eb390 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7ed52bb1aba48f1a0454fcb7a97c0a8'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:29.228776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 933/1905] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:28 2022] DELETE /identity/v3/users/a7ed52bb1aba48f1a0454fcb7a97c0a8 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:29.871984 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a701917-565e-4dd8-8c22-7b51de27627f admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:29.871984 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3a701917-565e-4dd8-8c22-7b51de27627f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:29.884083 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a701917-565e-4dd8-8c22-7b51de27627f admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:29.989341 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6fac5f-ff52-42b7-b189-9b88406ef01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:29.990868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fac5f-ff52-42b7-b189-9b88406ef01a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:29.991354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fac5f-ff52-42b7-b189-9b88406ef01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:29.991879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fac5f-ff52-42b7-b189-9b88406ef01a admin admin] PATH_INFO: `/v3/projects/94491c37393f444880c6456030ae19c9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:29.994006 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6fac5f-ff52-42b7-b189-9b88406ef01a admin admin] RBAC: Authorizing `identity:delete_project(project_id=94491c37393f444880c6456030ae19c9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:29.998187 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6fac5f-ff52-42b7-b189-9b88406ef01a admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:30.144836 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2c6fac5f-ff52-42b7-b189-9b88406ef01a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94491c37393f444880c6456030ae19c9', 'request_id': 'req-2c6fac5f-ff52-42b7-b189-9b88406ef01a'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:30.153703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2c6fac5f-ff52-42b7-b189-9b88406ef01a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94491c37393f444880c6456030ae19c9', 'request_id': 'req-2c6fac5f-ff52-42b7-b189-9b88406ef01a'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:30.161727 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2c6fac5f-ff52-42b7-b189-9b88406ef01a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94491c37393f444880c6456030ae19c9', 'request_id': 'req-2c6fac5f-ff52-42b7-b189-9b88406ef01a'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:30.255189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2c6fac5f-ff52-42b7-b189-9b88406ef01a admin admin] The token cache is being invalidate because project 94491c37393f444880c6456030ae19c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:30.255489 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2c6fac5f-ff52-42b7-b189-9b88406ef01a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:30.256230 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2c6fac5f-ff52-42b7-b189-9b88406ef01a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:30.262059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 973/1906] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:29 2022] DELETE /identity/v3/projects/94491c37393f444880c6456030ae19c9 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:30.263500 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7523d-366a-4b28-a336-8ba83afe3755 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:30.263956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7523d-366a-4b28-a336-8ba83afe3755 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:30.264092 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7523d-366a-4b28-a336-8ba83afe3755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:30.316545 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-fec7523d-366a-4b28-a336-8ba83afe3755 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:30.317817 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-fec7523d-366a-4b28-a336-8ba83afe3755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:30.362689 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:30.483809 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 934/1907] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:30.490854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5aa18e-e1f2-48eb-b307-b5bb1a2281e8 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:30.491222 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5aa18e-e1f2-48eb-b307-b5bb1a2281e8 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:30.491582 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5aa18e-e1f2-48eb-b307-b5bb1a2281e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:30.537799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-4b5aa18e-e1f2-48eb-b307-b5bb1a2281e8 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:30.539060 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-4b5aa18e-e1f2-48eb-b307-b5bb1a2281e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:30.551357 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 974/1908] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:30.566526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7523d-366a-4b28-a336-8ba83afe3755 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:30.583710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7523d-366a-4b28-a336-8ba83afe3755 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:30.592336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3da68-02ae-4920-9b7c-0c1a5e5cd02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:30.593260 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3da68-02ae-4920-9b7c-0c1a5e5cd02b admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:30.593658 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3da68-02ae-4920-9b7c-0c1a5e5cd02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:30.594031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3da68-02ae-4920-9b7c-0c1a5e5cd02b admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:30.594860 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df3da68-02ae-4920-9b7c-0c1a5e5cd02b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:46:30.595238 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df3da68-02ae-4920-9b7c-0c1a5e5cd02b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:30.597601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df3da68-02ae-4920-9b7c-0c1a5e5cd02b admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:30.628483 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 935/1909] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:46:30.638660 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5aa18e-e1f2-48eb-b307-b5bb1a2281e8 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:30.655258 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5aa18e-e1f2-48eb-b307-b5bb1a2281e8 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:30.662309 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56850a-2952-41f2-82b4-0b7aeba0b310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:30.663167 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56850a-2952-41f2-82b4-0b7aeba0b310 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:30.663386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56850a-2952-41f2-82b4-0b7aeba0b310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:30.663916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56850a-2952-41f2-82b4-0b7aeba0b310 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:30.665723 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a56850a-2952-41f2-82b4-0b7aeba0b310 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:30.668445 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a56850a-2952-41f2-82b4-0b7aeba0b310 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:30.713184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:30.827487 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 975/1910] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:30 2022] POST /identity/v3/projects => generated 393 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:30.827921 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3da68-02ae-4920-9b7c-0c1a5e5cd02b admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:30.840214 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3da68-02ae-4920-9b7c-0c1a5e5cd02b admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:30.923113 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9e386-4810-4a72-941e-3bcdf639514c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:30.924046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9e386-4810-4a72-941e-3bcdf639514c admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:30.924511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9e386-4810-4a72-941e-3bcdf639514c admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:30.924961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9e386-4810-4a72-941e-3bcdf639514c admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:30.925844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9e386-4810-4a72-941e-3bcdf639514c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:30.927833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9e386-4810-4a72-941e-3bcdf639514c admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:31.013392 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 936/1911] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:30 2022] POST /identity/v3/users => generated 297 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:46:31.018450 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56850a-2952-41f2-82b4-0b7aeba0b310 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:31.031710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56850a-2952-41f2-82b4-0b7aeba0b310 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:31.038630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842605f7-d321-4ec6-9765-afc685096556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:31.039402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842605f7-d321-4ec6-9765-afc685096556 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:31.046400 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842605f7-d321-4ec6-9765-afc685096556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:31.046400 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842605f7-d321-4ec6-9765-afc685096556 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:31.046400 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842605f7-d321-4ec6-9765-afc685096556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:31.046400 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842605f7-d321-4ec6-9765-afc685096556 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:31.055153 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 976/1912] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:31.059767 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9e386-4810-4a72-941e-3bcdf639514c admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:31.070489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9e386-4810-4a72-941e-3bcdf639514c admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:31.075810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e75301-feff-473a-8fde-7c151a12108e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:31.076475 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e75301-feff-473a-8fde-7c151a12108e admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:31.076791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e75301-feff-473a-8fde-7c151a12108e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:31.077069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e75301-feff-473a-8fde-7c151a12108e admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:31.077791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e75301-feff-473a-8fde-7c151a12108e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:31.079452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e75301-feff-473a-8fde-7c151a12108e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:31.091084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 937/1913] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:46:31.095125 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842605f7-d321-4ec6-9765-afc685096556 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:31.105832 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842605f7-d321-4ec6-9765-afc685096556 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:31.110529 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e161e-e0e6-4e0c-8734-6c8ccaaf1c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:31.111408 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e161e-e0e6-4e0c-8734-6c8ccaaf1c9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:31.111692 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e161e-e0e6-4e0c-8734-6c8ccaaf1c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:31.111969 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e161e-e0e6-4e0c-8734-6c8ccaaf1c9e admin admin] PATH_INFO: `/v3/projects/87b6152d7e574439a354f020eb1fcfdd/users/2778f510580242bfa31f65892569bf2b/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:31.131806 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e161e-e0e6-4e0c-8734-6c8ccaaf1c9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=87b6152d7e574439a354f020eb1fcfdd, user_id=2778f510580242bfa31f65892569bf2b, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:31.136248 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e161e-e0e6-4e0c-8734-6c8ccaaf1c9e admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:31.142787 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e75301-feff-473a-8fde-7c151a12108e admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:31.144601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-30e75301-feff-473a-8fde-7c151a12108e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:31.159982 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e75301-feff-473a-8fde-7c151a12108e admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:31.165449 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7deed-c5ee-427d-9868-68e5296e904b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:31.166340 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7deed-c5ee-427d-9868-68e5296e904b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:31.166627 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7deed-c5ee-427d-9868-68e5296e904b admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:31.166911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7deed-c5ee-427d-9868-68e5296e904b admin admin] PATH_INFO: `/v3/users/56b27cbf1e7149bd81a60bcb1e504670` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:31.168135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7deed-c5ee-427d-9868-68e5296e904b admin admin] RBAC: Authorizing `identity:delete_user(user_id=56b27cbf1e7149bd81a60bcb1e504670)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:31.170318 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7deed-c5ee-427d-9868-68e5296e904b admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:31.236460 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 977/1914] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:31 2022] PUT /identity/v3/projects/87b6152d7e574439a354f020eb1fcfdd/users/2778f510580242bfa31f65892569bf2b/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:46:31.248805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09319185-7e48-457a-ad4e-53f4e596122a None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:31.249172 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09319185-7e48-457a-ad4e-53f4e596122a None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:31.249753 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09319185-7e48-457a-ad4e-53f4e596122a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:31.280082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-09319185-7e48-457a-ad4e-53f4e596122a None None] MFA Rules not processed for user `2778f510580242bfa31f65892569bf2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:31.281327 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-09319185-7e48-457a-ad4e-53f4e596122a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:31.366848 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 978/1915] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:46:31.373367 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09319185-7e48-457a-ad4e-53f4e596122a None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:31.376526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-eca7deed-c5ee-427d-9868-68e5296e904b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56b27cbf1e7149bd81a60bcb1e504670'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:31.406806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-eca7deed-c5ee-427d-9868-68e5296e904b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56b27cbf1e7149bd81a60bcb1e504670'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:31.416929 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:31.468035 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09319185-7e48-457a-ad4e-53f4e596122a None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:31.473554 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f5ab3-da46-4c26-81b7-2cfd9f048d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:31.474995 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f5ab3-da46-4c26-81b7-2cfd9f048d4b admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:31.475443 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f5ab3-da46-4c26-81b7-2cfd9f048d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:31.476433 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f5ab3-da46-4c26-81b7-2cfd9f048d4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:31.477285 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f5ab3-da46-4c26-81b7-2cfd9f048d4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:31.480912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f5ab3-da46-4c26-81b7-2cfd9f048d4b admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:31.520888 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-eca7deed-c5ee-427d-9868-68e5296e904b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56b27cbf1e7149bd81a60bcb1e504670'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:31.522126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-eca7deed-c5ee-427d-9868-68e5296e904b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56b27cbf1e7149bd81a60bcb1e504670'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:31.522850 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-eca7deed-c5ee-427d-9868-68e5296e904b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56b27cbf1e7149bd81a60bcb1e504670'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:31.548320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 938/1916] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:31 2022] DELETE /identity/v3/users/56b27cbf1e7149bd81a60bcb1e504670 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:31.553316 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d5b1c-3582-4469-b6b0-d96ab1ce18aa None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:31.553625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d5b1c-3582-4469-b6b0-d96ab1ce18aa None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:31.553915 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d5b1c-3582-4469-b6b0-d96ab1ce18aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:31.577256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-aa8d5b1c-3582-4469-b6b0-d96ab1ce18aa None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:31.578886 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-aa8d5b1c-3582-4469-b6b0-d96ab1ce18aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:31.580762 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 979/1917] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:31 2022] POST /identity/v3/projects => generated 391 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:31.584715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f5ab3-da46-4c26-81b7-2cfd9f048d4b admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:31.589237 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-945f5ab3-da46-4c26-81b7-2cfd9f048d4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:31.673584 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:31.711401 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 939/1918] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:46:31.721154 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d5b1c-3582-4469-b6b0-d96ab1ce18aa None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:31.723538 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-aa8d5b1c-3582-4469-b6b0-d96ab1ce18aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:31.741995 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f5ab3-da46-4c26-81b7-2cfd9f048d4b admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:31.747499 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb0d1a-60cd-4df3-9fd8-65efe8c691d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:31.748193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb0d1a-60cd-4df3-9fd8-65efe8c691d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:31.748478 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb0d1a-60cd-4df3-9fd8-65efe8c691d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:31.749059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb0d1a-60cd-4df3-9fd8-65efe8c691d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:31.749758 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb0d1a-60cd-4df3-9fd8-65efe8c691d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:31.752446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb0d1a-60cd-4df3-9fd8-65efe8c691d6 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:31.797894 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d5b1c-3582-4469-b6b0-d96ab1ce18aa None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:31.801860 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50ec18-ba6e-4396-a39a-b6ddc06dc827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:31.802527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50ec18-ba6e-4396-a39a-b6ddc06dc827 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:31.802815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50ec18-ba6e-4396-a39a-b6ddc06dc827 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:31.803095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50ec18-ba6e-4396-a39a-b6ddc06dc827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:31.819354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50ec18-ba6e-4396-a39a-b6ddc06dc827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:31.821055 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50ec18-ba6e-4396-a39a-b6ddc06dc827 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:31.838836 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 940/1919] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:31.878588 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 980/1920] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:31 2022] POST /identity/v3/users => generated 302 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:46:31.882362 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50ec18-ba6e-4396-a39a-b6ddc06dc827 service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:31.892953 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50ec18-ba6e-4396-a39a-b6ddc06dc827 service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:31.898372 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111c77a-3ea6-4f50-b03d-b10707633164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:31.899156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111c77a-3ea6-4f50-b03d-b10707633164 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:31.900030 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111c77a-3ea6-4f50-b03d-b10707633164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:31.900255 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111c77a-3ea6-4f50-b03d-b10707633164 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:31.901128 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111c77a-3ea6-4f50-b03d-b10707633164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:31.903253 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111c77a-3ea6-4f50-b03d-b10707633164 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:31.921390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 941/1921] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:31.925301 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb0d1a-60cd-4df3-9fd8-65efe8c691d6 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:31.940636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb0d1a-60cd-4df3-9fd8-65efe8c691d6 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:31.951851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c305a3e-449a-4cc9-9491-425e2a33ec0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:31.953089 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c305a3e-449a-4cc9-9491-425e2a33ec0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:31.953517 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c305a3e-449a-4cc9-9491-425e2a33ec0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:31.953968 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c305a3e-449a-4cc9-9491-425e2a33ec0a admin admin] PATH_INFO: `/v3/projects/d6f9d6c8fcc3442db15483575c444739/users/cae23cde84d04abbbefdbfd5bfe9eb94/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:31.974483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c305a3e-449a-4cc9-9491-425e2a33ec0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6f9d6c8fcc3442db15483575c444739, user_id=cae23cde84d04abbbefdbfd5bfe9eb94, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:31.977016 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c305a3e-449a-4cc9-9491-425e2a33ec0a admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:32.062207 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 981/1922] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:31 2022] PUT /identity/v3/projects/d6f9d6c8fcc3442db15483575c444739/users/cae23cde84d04abbbefdbfd5bfe9eb94/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:32.066388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111c77a-3ea6-4f50-b03d-b10707633164 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:32.080040 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111c77a-3ea6-4f50-b03d-b10707633164 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:32.228666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa50a7-b83d-45f5-a12f-e12896903732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:32.229357 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa50a7-b83d-45f5-a12f-e12896903732 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:32.229652 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa50a7-b83d-45f5-a12f-e12896903732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:32.229969 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa50a7-b83d-45f5-a12f-e12896903732 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:32.230732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa50a7-b83d-45f5-a12f-e12896903732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:32.234642 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa50a7-b83d-45f5-a12f-e12896903732 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:32.256671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 942/1923] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:32.260571 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c305a3e-449a-4cc9-9491-425e2a33ec0a admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:32.274976 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c305a3e-449a-4cc9-9491-425e2a33ec0a admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:32.281459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06babe-e59f-4234-89f4-6b82d45c0820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:32.282667 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06babe-e59f-4234-89f4-6b82d45c0820 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:32.283159 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06babe-e59f-4234-89f4-6b82d45c0820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:32.283626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06babe-e59f-4234-89f4-6b82d45c0820 admin admin] PATH_INFO: `/v3/projects/d6f9d6c8fcc3442db15483575c444739/users/cae23cde84d04abbbefdbfd5bfe9eb94/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:32.291393 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06babe-e59f-4234-89f4-6b82d45c0820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6f9d6c8fcc3442db15483575c444739, user_id=cae23cde84d04abbbefdbfd5bfe9eb94, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:32.295997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06babe-e59f-4234-89f4-6b82d45c0820 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:32.311690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 982/1924] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:32 2022] PUT /identity/v3/projects/d6f9d6c8fcc3442db15483575c444739/users/cae23cde84d04abbbefdbfd5bfe9eb94/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:32.322728 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34ef31-0411-4bf2-89b5-8d752138ed46 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:32.323028 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34ef31-0411-4bf2-89b5-8d752138ed46 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:32.323315 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34ef31-0411-4bf2-89b5-8d752138ed46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:32.365054 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-bf34ef31-0411-4bf2-89b5-8d752138ed46 None None] MFA Rules not processed for user `cae23cde84d04abbbefdbfd5bfe9eb94`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:32.366349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-bf34ef31-0411-4bf2-89b5-8d752138ed46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:32.369136 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06babe-e59f-4234-89f4-6b82d45c0820 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:32.372111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-0d06babe-e59f-4234-89f4-6b82d45c0820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:32.373203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:32.464152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 943/1925] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:32.470936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34ef31-0411-4bf2-89b5-8d752138ed46 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:32.473833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-bf34ef31-0411-4bf2-89b5-8d752138ed46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:32.589327 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34ef31-0411-4bf2-89b5-8d752138ed46 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:32.593259 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539c9a7-45bc-4fcb-8967-c42e8313812d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:32.594191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539c9a7-45bc-4fcb-8967-c42e8313812d service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:32.594424 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539c9a7-45bc-4fcb-8967-c42e8313812d service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:32.594824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539c9a7-45bc-4fcb-8967-c42e8313812d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:32.596600 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06babe-e59f-4234-89f4-6b82d45c0820 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:32.606475 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539c9a7-45bc-4fcb-8967-c42e8313812d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:32.608382 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539c9a7-45bc-4fcb-8967-c42e8313812d service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:32.624046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 944/1926] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:32 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:32.625543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3e88b-63a2-475a-9b56-214af6be9ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:32.627026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3e88b-63a2-475a-9b56-214af6be9ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:32.627623 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3e88b-63a2-475a-9b56-214af6be9ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:32.627918 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3e88b-63a2-475a-9b56-214af6be9ca5 admin admin] PATH_INFO: `/v3/projects/933706baf68c42f69bd41249d5c0faef` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:32.632932 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3e88b-63a2-475a-9b56-214af6be9ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=933706baf68c42f69bd41249d5c0faef)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:32.635249 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3e88b-63a2-475a-9b56-214af6be9ca5 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:32.722168 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dc6cd-4b7c-4eb5-a4b3-fc895538aa75 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:32.722336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dc6cd-4b7c-4eb5-a4b3-fc895538aa75 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:32.722620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dc6cd-4b7c-4eb5-a4b3-fc895538aa75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:32.772485 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-0f8dc6cd-4b7c-4eb5-a4b3-fc895538aa75 None None] MFA Rules not processed for user `84a5fb262284400591cf29dd13771ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:32.772485 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-0f8dc6cd-4b7c-4eb5-a4b3-fc895538aa75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:32.813772 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-1ac3e88b-63a2-475a-9b56-214af6be9ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '933706baf68c42f69bd41249d5c0faef', 'request_id': 'req-1ac3e88b-63a2-475a-9b56-214af6be9ca5'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:32.824166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-1ac3e88b-63a2-475a-9b56-214af6be9ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '933706baf68c42f69bd41249d5c0faef', 'request_id': 'req-1ac3e88b-63a2-475a-9b56-214af6be9ca5'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:32.849550 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-1ac3e88b-63a2-475a-9b56-214af6be9ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '933706baf68c42f69bd41249d5c0faef', 'request_id': 'req-1ac3e88b-63a2-475a-9b56-214af6be9ca5'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:32.854690 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 945/1927] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:46:32.855969 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8dc6cd-4b7c-4eb5-a4b3-fc895538aa75 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:32.898714 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:32.922064 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8dc6cd-4b7c-4eb5-a4b3-fc895538aa75 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:32.925942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a51509-3a61-41ab-8ada-a6006f91e484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:32.926602 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a51509-3a61-41ab-8ada-a6006f91e484 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:32.931495 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a51509-3a61-41ab-8ada-a6006f91e484 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:32.931495 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a51509-3a61-41ab-8ada-a6006f91e484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:32.931495 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-18a51509-3a61-41ab-8ada-a6006f91e484 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:32.933648 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-1ac3e88b-63a2-475a-9b56-214af6be9ca5 admin admin] The token cache is being invalidate because project 933706baf68c42f69bd41249d5c0faef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:32.934016 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-1ac3e88b-63a2-475a-9b56-214af6be9ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:32.934768 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-1ac3e88b-63a2-475a-9b56-214af6be9ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:32.936631 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 983/1928] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:32 2022] DELETE /identity/v3/projects/933706baf68c42f69bd41249d5c0faef => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:32.940899 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3e88b-63a2-475a-9b56-214af6be9ca5 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:32.942647 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-1ac3e88b-63a2-475a-9b56-214af6be9ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:32.972034 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:33.000193 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a51509-3a61-41ab-8ada-a6006f91e484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:33.002197 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a51509-3a61-41ab-8ada-a6006f91e484 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:33.015212 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 946/1929] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:33.016522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a51509-3a61-41ab-8ada-a6006f91e484 service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:33.043281 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a51509-3a61-41ab-8ada-a6006f91e484 service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:33.047217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3e88b-63a2-475a-9b56-214af6be9ca5 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:33.053492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71153c-5001-474f-8796-8a1033be3101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:33.058061 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71153c-5001-474f-8796-8a1033be3101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:33.058061 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71153c-5001-474f-8796-8a1033be3101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:33.058061 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71153c-5001-474f-8796-8a1033be3101 admin admin] PATH_INFO: `/v3/users/4ac94628e02c4e0aaa02149ff9707229` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:33.058061 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f71153c-5001-474f-8796-8a1033be3101 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ac94628e02c4e0aaa02149ff9707229)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:33.060528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f71153c-5001-474f-8796-8a1033be3101 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:33.061752 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b3a03-9a08-4858-b69a-ef4672cf1713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:33.062589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b3a03-9a08-4858-b69a-ef4672cf1713 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:33.063161 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b3a03-9a08-4858-b69a-ef4672cf1713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:33.063161 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b3a03-9a08-4858-b69a-ef4672cf1713 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:33.063826 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b3a03-9a08-4858-b69a-ef4672cf1713 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:33.065799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b3a03-9a08-4858-b69a-ef4672cf1713 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:33.209144 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 984/1930] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:32 2022] POST /identity/v3/projects => generated 379 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:33.210071 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b3a03-9a08-4858-b69a-ef4672cf1713 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:33.278869 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b3a03-9a08-4858-b69a-ef4672cf1713 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:33.286794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd014d-296d-4bfb-b1ac-2ed25bb2e2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:33.287492 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd014d-296d-4bfb-b1ac-2ed25bb2e2c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:33.287793 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd014d-296d-4bfb-b1ac-2ed25bb2e2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:33.291639 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd014d-296d-4bfb-b1ac-2ed25bb2e2c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:33.291639 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd014d-296d-4bfb-b1ac-2ed25bb2e2c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:33.291639 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd014d-296d-4bfb-b1ac-2ed25bb2e2c3 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:33.304392 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-8f71153c-5001-474f-8796-8a1033be3101 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ac94628e02c4e0aaa02149ff9707229'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:33.319358 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-8f71153c-5001-474f-8796-8a1033be3101 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ac94628e02c4e0aaa02149ff9707229'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:33.366693 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 985/1931] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:33 2022] POST /identity/v3/users => generated 296 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:33.373775 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd014d-296d-4bfb-b1ac-2ed25bb2e2c3 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:33.385233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-8f71153c-5001-474f-8796-8a1033be3101 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ac94628e02c4e0aaa02149ff9707229'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:33.385821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd014d-296d-4bfb-b1ac-2ed25bb2e2c3 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:33.386344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-8f71153c-5001-474f-8796-8a1033be3101 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ac94628e02c4e0aaa02149ff9707229'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:33.387065 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-8f71153c-5001-474f-8796-8a1033be3101 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ac94628e02c4e0aaa02149ff9707229'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:33.391061 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b66f8-392c-4943-8f99-15443a75d3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:33.394924 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b66f8-392c-4943-8f99-15443a75d3c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:33.394924 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b66f8-392c-4943-8f99-15443a75d3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:33.394924 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b66f8-392c-4943-8f99-15443a75d3c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:33.394924 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b66f8-392c-4943-8f99-15443a75d3c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:33.395664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b66f8-392c-4943-8f99-15443a75d3c7 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:33.405637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 947/1932] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:33 2022] DELETE /identity/v3/users/4ac94628e02c4e0aaa02149ff9707229 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:33.407520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 986/1933] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:46:33.411429 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71153c-5001-474f-8796-8a1033be3101 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:33.412840 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-8f71153c-5001-474f-8796-8a1033be3101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:33.485940 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71153c-5001-474f-8796-8a1033be3101 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:33.491179 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820ece3-8840-483e-a24d-1cb465dd461f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:33.492251 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820ece3-8840-483e-a24d-1cb465dd461f admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:33.492653 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820ece3-8840-483e-a24d-1cb465dd461f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:33.492940 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820ece3-8840-483e-a24d-1cb465dd461f admin admin] PATH_INFO: `/v3/projects/a2afecbf44e543fd89fafe08a7ec7a01/users/e2e0900058f7442fa48c48903781317c/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:33.496292 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:33.510198 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820ece3-8840-483e-a24d-1cb465dd461f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2afecbf44e543fd89fafe08a7ec7a01, user_id=e2e0900058f7442fa48c48903781317c, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:33.513210 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820ece3-8840-483e-a24d-1cb465dd461f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:33.617839 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 948/1934] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:33 2022] PUT /identity/v3/projects/a2afecbf44e543fd89fafe08a7ec7a01/users/e2e0900058f7442fa48c48903781317c/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:33.625501 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b66f8-392c-4943-8f99-15443a75d3c7 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:33.636478 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b66f8-392c-4943-8f99-15443a75d3c7 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:33.721153 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52a415-c60a-4da7-abaa-9f68fa878f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:33.721538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52a415-c60a-4da7-abaa-9f68fa878f60 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:33.721915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52a415-c60a-4da7-abaa-9f68fa878f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:33.722257 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52a415-c60a-4da7-abaa-9f68fa878f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:33.723079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52a415-c60a-4da7-abaa-9f68fa878f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:33.724878 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52a415-c60a-4da7-abaa-9f68fa878f60 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:33.739664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 987/1935] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:33.740828 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820ece3-8840-483e-a24d-1cb465dd461f admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:33.754406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820ece3-8840-483e-a24d-1cb465dd461f admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:33.759837 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa912933-a8b8-442d-8182-edf8ce9ec9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:33.760761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa912933-a8b8-442d-8182-edf8ce9ec9d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:33.761046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa912933-a8b8-442d-8182-edf8ce9ec9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:33.766162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa912933-a8b8-442d-8182-edf8ce9ec9d2 admin admin] PATH_INFO: `/v3/projects/a2afecbf44e543fd89fafe08a7ec7a01/users/e2e0900058f7442fa48c48903781317c/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:33.766162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa912933-a8b8-442d-8182-edf8ce9ec9d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2afecbf44e543fd89fafe08a7ec7a01, user_id=e2e0900058f7442fa48c48903781317c, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:33.766162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa912933-a8b8-442d-8182-edf8ce9ec9d2 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:33.777249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 949/1936] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:33 2022] PUT /identity/v3/projects/a2afecbf44e543fd89fafe08a7ec7a01/users/e2e0900058f7442fa48c48903781317c/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:46:33.884783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52a415-c60a-4da7-abaa-9f68fa878f60 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:33.887650 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-1f52a415-c60a-4da7-abaa-9f68fa878f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:33.907175 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52a415-c60a-4da7-abaa-9f68fa878f60 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:33.928710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:34.004304 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69fc97-bae3-4a18-aee8-f6f726bf39cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:34.005540 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69fc97-bae3-4a18-aee8-f6f726bf39cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:34.006589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69fc97-bae3-4a18-aee8-f6f726bf39cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:34.006904 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69fc97-bae3-4a18-aee8-f6f726bf39cd admin admin] PATH_INFO: `/v3/projects/a96b14fe70754624806e9252d5ebc21c` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:34.008167 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69fc97-bae3-4a18-aee8-f6f726bf39cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a96b14fe70754624806e9252d5ebc21c)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:34.011072 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69fc97-bae3-4a18-aee8-f6f726bf39cd admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:34.168093 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cc69fc97-bae3-4a18-aee8-f6f726bf39cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a96b14fe70754624806e9252d5ebc21c', 'request_id': 'req-cc69fc97-bae3-4a18-aee8-f6f726bf39cd'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:34.176049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cc69fc97-bae3-4a18-aee8-f6f726bf39cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a96b14fe70754624806e9252d5ebc21c', 'request_id': 'req-cc69fc97-bae3-4a18-aee8-f6f726bf39cd'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:34.183013 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cc69fc97-bae3-4a18-aee8-f6f726bf39cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a96b14fe70754624806e9252d5ebc21c', 'request_id': 'req-cc69fc97-bae3-4a18-aee8-f6f726bf39cd'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:34.263988 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cc69fc97-bae3-4a18-aee8-f6f726bf39cd admin admin] The token cache is being invalidate because project a96b14fe70754624806e9252d5ebc21c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:34.264395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cc69fc97-bae3-4a18-aee8-f6f726bf39cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:34.265870 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cc69fc97-bae3-4a18-aee8-f6f726bf39cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:34.273974 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 988/1937] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:33 2022] DELETE /identity/v3/projects/a96b14fe70754624806e9252d5ebc21c => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:34.279913 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5119e9-173f-48f9-a559-ab0f28f69b5b None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:34.280212 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5119e9-173f-48f9-a559-ab0f28f69b5b None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:34.280503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5119e9-173f-48f9-a559-ab0f28f69b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:34.307763 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-8b5119e9-173f-48f9-a559-ab0f28f69b5b None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:34.309416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-8b5119e9-173f-48f9-a559-ab0f28f69b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:34.407182 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 950/1938] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:34.408338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bd944-8767-4bfb-911d-9f1822883340 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:34.408663 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bd944-8767-4bfb-911d-9f1822883340 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:34.408945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bd944-8767-4bfb-911d-9f1822883340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:34.439553 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-436bd944-8767-4bfb-911d-9f1822883340 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:34.441999 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-436bd944-8767-4bfb-911d-9f1822883340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:34.493260 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 989/1939] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:46:34.500707 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5119e9-173f-48f9-a559-ab0f28f69b5b None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:34.516358 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5119e9-173f-48f9-a559-ab0f28f69b5b None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:34.535342 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f28264-2042-4234-948b-d52916d728b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:34.536338 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f28264-2042-4234-948b-d52916d728b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:34.536712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f28264-2042-4234-948b-d52916d728b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:34.537036 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f28264-2042-4234-948b-d52916d728b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:34.543646 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f28264-2042-4234-948b-d52916d728b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:46:34.543646 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f28264-2042-4234-948b-d52916d728b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:34.543646 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f28264-2042-4234-948b-d52916d728b0 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:34.568673 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 951/1940] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:46:34.570071 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436bd944-8767-4bfb-911d-9f1822883340 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:34.581443 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436bd944-8767-4bfb-911d-9f1822883340 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:34.586929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6b3d0-830b-47b2-9215-2096d1336a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:34.588268 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6b3d0-830b-47b2-9215-2096d1336a8e admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:34.588610 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6b3d0-830b-47b2-9215-2096d1336a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:34.588933 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6b3d0-830b-47b2-9215-2096d1336a8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:34.589842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6b3d0-830b-47b2-9215-2096d1336a8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:34.591749 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6b3d0-830b-47b2-9215-2096d1336a8e admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:34.701542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 990/1941] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:34 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:34.705513 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f28264-2042-4234-948b-d52916d728b0 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:34.722060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f28264-2042-4234-948b-d52916d728b0 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:34.815751 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1eb043-cc9d-41e7-849f-3f0febc245f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:34.816466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1eb043-cc9d-41e7-849f-3f0febc245f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:34.816752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1eb043-cc9d-41e7-849f-3f0febc245f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:34.820684 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1eb043-cc9d-41e7-849f-3f0febc245f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:34.820684 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1eb043-cc9d-41e7-849f-3f0febc245f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:34.820684 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1eb043-cc9d-41e7-849f-3f0febc245f8 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:34.907940 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 952/1942] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:34 2022] POST /identity/v3/users => generated 289 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:34.911903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6b3d0-830b-47b2-9215-2096d1336a8e admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:35.160933 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6b3d0-830b-47b2-9215-2096d1336a8e admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:35.160933 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a367e4e-54ab-446f-9dd5-26f867157bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:35.160933 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a367e4e-54ab-446f-9dd5-26f867157bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:35.160933 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a367e4e-54ab-446f-9dd5-26f867157bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:35.197608 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a367e4e-54ab-446f-9dd5-26f867157bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:35.197608 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a367e4e-54ab-446f-9dd5-26f867157bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:35.197608 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a367e4e-54ab-446f-9dd5-26f867157bb1 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:35.197608 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 991/1943] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:35.198753 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1eb043-cc9d-41e7-849f-3f0febc245f8 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:35.198753 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1eb043-cc9d-41e7-849f-3f0febc245f8 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:35.198753 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b6de7-d28c-4e56-8214-46b58c68be84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:35.198753 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b6de7-d28c-4e56-8214-46b58c68be84 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:35.199161 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b6de7-d28c-4e56-8214-46b58c68be84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:35.199161 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b6de7-d28c-4e56-8214-46b58c68be84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:35.199161 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b6de7-d28c-4e56-8214-46b58c68be84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:35.199161 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b6de7-d28c-4e56-8214-46b58c68be84 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:35.199161 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 953/1944] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:35.199522 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a367e4e-54ab-446f-9dd5-26f867157bb1 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:35.199522 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a367e4e-54ab-446f-9dd5-26f867157bb1 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:35.199522 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512872b-575a-427b-b8f3-593ed5285400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:35.199522 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512872b-575a-427b-b8f3-593ed5285400 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:35.199812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512872b-575a-427b-b8f3-593ed5285400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:35.199812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512872b-575a-427b-b8f3-593ed5285400 admin admin] PATH_INFO: `/v3/projects/825b91d2627543fbb7ce3c5a4f77b7de/users/d82614c2d5d64e9ab815b86daa0fd105/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:35.199812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5512872b-575a-427b-b8f3-593ed5285400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=825b91d2627543fbb7ce3c5a4f77b7de, user_id=d82614c2d5d64e9ab815b86daa0fd105, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:35.199812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5512872b-575a-427b-b8f3-593ed5285400 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:35.230309 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 992/1945] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:35 2022] PUT /identity/v3/projects/825b91d2627543fbb7ce3c5a4f77b7de/users/d82614c2d5d64e9ab815b86daa0fd105/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:35.241721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2346a19-c875-49c0-92cf-6e78eecd82cb None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:35.242038 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2346a19-c875-49c0-92cf-6e78eecd82cb None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:35.242345 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2346a19-c875-49c0-92cf-6e78eecd82cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:35.300123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-a2346a19-c875-49c0-92cf-6e78eecd82cb None None] MFA Rules not processed for user `d82614c2d5d64e9ab815b86daa0fd105`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:35.301660 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-a2346a19-c875-49c0-92cf-6e78eecd82cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:35.331961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:35.388836 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 954/1946] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:35.393525 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512872b-575a-427b-b8f3-593ed5285400 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:35.450161 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:35.478200 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512872b-575a-427b-b8f3-593ed5285400 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:35.485406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f71fe7-3ca9-4733-a868-169ae256cf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:35.486746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f71fe7-3ca9-4733-a868-169ae256cf39 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:35.487194 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f71fe7-3ca9-4733-a868-169ae256cf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:35.487644 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f71fe7-3ca9-4733-a868-169ae256cf39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:35.488566 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f71fe7-3ca9-4733-a868-169ae256cf39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:35.490642 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f71fe7-3ca9-4733-a868-169ae256cf39 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:35.627783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 993/1947] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:35 2022] POST /identity/v3/projects => generated 381 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:35.629125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2346a19-c875-49c0-92cf-6e78eecd82cb None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:35.644922 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2346a19-c875-49c0-92cf-6e78eecd82cb None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:35.742552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c9f8c-e942-4da3-a386-e4e85ff56a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:35.743695 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c9f8c-e942-4da3-a386-e4e85ff56a98 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:35.744561 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c9f8c-e942-4da3-a386-e4e85ff56a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:35.745037 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c9f8c-e942-4da3-a386-e4e85ff56a98 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:35.746275 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c9f8c-e942-4da3-a386-e4e85ff56a98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:35.749209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c9f8c-e942-4da3-a386-e4e85ff56a98 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:35.830505 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 955/1948] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:35 2022] POST /identity/v3/users => generated 297 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:35.833250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f71fe7-3ca9-4733-a868-169ae256cf39 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:35.865393 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f71fe7-3ca9-4733-a868-169ae256cf39 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:35.890497 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83916ef-f62f-46ce-a162-42d3e7811185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:35.895217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83916ef-f62f-46ce-a162-42d3e7811185 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:35.895563 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83916ef-f62f-46ce-a162-42d3e7811185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:35.901140 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83916ef-f62f-46ce-a162-42d3e7811185 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:35.902999 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83916ef-f62f-46ce-a162-42d3e7811185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:35.911122 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83916ef-f62f-46ce-a162-42d3e7811185 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:35.969629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 994/1949] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:35.970694 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c9f8c-e942-4da3-a386-e4e85ff56a98 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:35.989605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c9f8c-e942-4da3-a386-e4e85ff56a98 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:36.004424 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5933cf-1552-4201-9891-5d9dfa684c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:36.005373 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5933cf-1552-4201-9891-5d9dfa684c46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:36.005661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5933cf-1552-4201-9891-5d9dfa684c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:36.006004 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5933cf-1552-4201-9891-5d9dfa684c46 admin admin] PATH_INFO: `/v3/projects/8a15d7d5005b49e7b86057e09ca2883f/users/d084bb37aa2544ebb541ddf5b73fe892/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:36.045659 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5933cf-1552-4201-9891-5d9dfa684c46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a15d7d5005b49e7b86057e09ca2883f, user_id=d084bb37aa2544ebb541ddf5b73fe892, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:36.048439 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5933cf-1552-4201-9891-5d9dfa684c46 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:36.177141 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 956/1950] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:35 2022] PUT /identity/v3/projects/8a15d7d5005b49e7b86057e09ca2883f/users/d084bb37aa2544ebb541ddf5b73fe892/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:36.179428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83916ef-f62f-46ce-a162-42d3e7811185 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:36.205489 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83916ef-f62f-46ce-a162-42d3e7811185 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:36.328351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019c682-5b5f-46e1-a26d-e9c7b5a2fda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:36.329085 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019c682-5b5f-46e1-a26d-e9c7b5a2fda8 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:36.329376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019c682-5b5f-46e1-a26d-e9c7b5a2fda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:36.329663 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019c682-5b5f-46e1-a26d-e9c7b5a2fda8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:36.331415 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019c682-5b5f-46e1-a26d-e9c7b5a2fda8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:36.335276 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019c682-5b5f-46e1-a26d-e9c7b5a2fda8 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:36.360710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 995/1951] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:36 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:36.361358 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5933cf-1552-4201-9891-5d9dfa684c46 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:36.375574 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5933cf-1552-4201-9891-5d9dfa684c46 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:36.410668 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c1a32-fe48-4878-80ad-e401590c48fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:36.412941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c1a32-fe48-4878-80ad-e401590c48fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:36.413541 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c1a32-fe48-4878-80ad-e401590c48fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:36.415726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c1a32-fe48-4878-80ad-e401590c48fa admin admin] PATH_INFO: `/v3/projects/8a15d7d5005b49e7b86057e09ca2883f/users/d084bb37aa2544ebb541ddf5b73fe892/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:36.418979 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c1a32-fe48-4878-80ad-e401590c48fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a15d7d5005b49e7b86057e09ca2883f, user_id=d084bb37aa2544ebb541ddf5b73fe892, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:36.424168 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c1a32-fe48-4878-80ad-e401590c48fa admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:36.443861 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 957/1952] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:36 2022] PUT /identity/v3/projects/8a15d7d5005b49e7b86057e09ca2883f/users/d084bb37aa2544ebb541ddf5b73fe892/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:46:36.455104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d9ca2-ccff-41eb-a375-a1ae616d615d None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:36.456931 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d9ca2-ccff-41eb-a375-a1ae616d615d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:36.456931 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d9ca2-ccff-41eb-a375-a1ae616d615d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:36.552234 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-488d9ca2-ccff-41eb-a375-a1ae616d615d None None] MFA Rules not processed for user `d084bb37aa2544ebb541ddf5b73fe892`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:36.555038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-488d9ca2-ccff-41eb-a375-a1ae616d615d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:36.743513 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 996/1953] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:36.771343 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c1a32-fe48-4878-80ad-e401590c48fa admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:36.771343 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-265c1a32-fe48-4878-80ad-e401590c48fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:36.855953 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c1a32-fe48-4878-80ad-e401590c48fa admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:36.861077 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe79df-e87c-47cf-8376-e36a839d7194 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:36.862105 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe79df-e87c-47cf-8376-e36a839d7194 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:36.862449 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe79df-e87c-47cf-8376-e36a839d7194 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:36.862799 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe79df-e87c-47cf-8376-e36a839d7194 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:36.875521 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe79df-e87c-47cf-8376-e36a839d7194 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:36.881257 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe79df-e87c-47cf-8376-e36a839d7194 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:36.891065 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 958/1954] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:37.049309 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d9ca2-ccff-41eb-a375-a1ae616d615d None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:37.050859 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-488d9ca2-ccff-41eb-a375-a1ae616d615d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:37.119733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:37.146432 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d9ca2-ccff-41eb-a375-a1ae616d615d None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:37.154344 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcfa63-6e1b-429a-99f6-5714fe9e411e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:37.155337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcfa63-6e1b-429a-99f6-5714fe9e411e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:37.155699 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcfa63-6e1b-429a-99f6-5714fe9e411e admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:37.156036 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcfa63-6e1b-429a-99f6-5714fe9e411e admin admin] PATH_INFO: `/v3/users/a8fe3d4a894b49f3b22515bd77b1ae4c` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:37.157407 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bcfa63-6e1b-429a-99f6-5714fe9e411e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8fe3d4a894b49f3b22515bd77b1ae4c)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:37.159509 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bcfa63-6e1b-429a-99f6-5714fe9e411e admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:37.410116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-43bcfa63-6e1b-429a-99f6-5714fe9e411e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8fe3d4a894b49f3b22515bd77b1ae4c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:37.429190 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-43bcfa63-6e1b-429a-99f6-5714fe9e411e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8fe3d4a894b49f3b22515bd77b1ae4c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:37.430002 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-43bcfa63-6e1b-429a-99f6-5714fe9e411e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8fe3d4a894b49f3b22515bd77b1ae4c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:37.430738 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-43bcfa63-6e1b-429a-99f6-5714fe9e411e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8fe3d4a894b49f3b22515bd77b1ae4c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:37.439863 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-43bcfa63-6e1b-429a-99f6-5714fe9e411e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8fe3d4a894b49f3b22515bd77b1ae4c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:37.541050 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 997/1955] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:37 2022] DELETE /identity/v3/users/a8fe3d4a894b49f3b22515bd77b1ae4c => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:37.547320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f91e3-2936-4e9a-ac26-6943e6ffe624 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:37.547663 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f91e3-2936-4e9a-ac26-6943e6ffe624 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:37.547902 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f91e3-2936-4e9a-ac26-6943e6ffe624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:37.600375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-8a2f91e3-2936-4e9a-ac26-6943e6ffe624 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:37.601662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-8a2f91e3-2936-4e9a-ac26-6943e6ffe624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:37.622185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:37.695266 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 959/1956] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:37.707246 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcfa63-6e1b-429a-99f6-5714fe9e411e admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:37.709223 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-43bcfa63-6e1b-429a-99f6-5714fe9e411e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:37.785104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcfa63-6e1b-429a-99f6-5714fe9e411e admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:37.785104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65127c-c7c4-40c0-8494-9d5d12f08893 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:37.785104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65127c-c7c4-40c0-8494-9d5d12f08893 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:37.785548 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65127c-c7c4-40c0-8494-9d5d12f08893 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:37.785628 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65127c-c7c4-40c0-8494-9d5d12f08893 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:37.796831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65127c-c7c4-40c0-8494-9d5d12f08893 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:37.798532 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65127c-c7c4-40c0-8494-9d5d12f08893 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:37.812269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 998/1957] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:38.002194 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f91e3-2936-4e9a-ac26-6943e6ffe624 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:38.005205 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-8a2f91e3-2936-4e9a-ac26-6943e6ffe624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:38.101476 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f91e3-2936-4e9a-ac26-6943e6ffe624 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:38.111506 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6772dd5-6d76-482a-bcd1-80aaa12b01fc tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd084bb37aa2544ebb541ddf5b73fe892', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a15d7d5005b49e7b86057e09ca2883f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:38.112641 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6772dd5-6d76-482a-bcd1-80aaa12b01fc tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:38.112641 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6772dd5-6d76-482a-bcd1-80aaa12b01fc tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:38.113721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6772dd5-6d76-482a-bcd1-80aaa12b01fc tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:38.114581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6772dd5-6d76-482a-bcd1-80aaa12b01fc tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:46:38.114881 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6772dd5-6d76-482a-bcd1-80aaa12b01fc tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:38.116525 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6772dd5-6d76-482a-bcd1-80aaa12b01fc tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:38.131728 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 960/1958] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:46:38.141733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65127c-c7c4-40c0-8494-9d5d12f08893 service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:38.150611 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65127c-c7c4-40c0-8494-9d5d12f08893 service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:38.156376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21788a45-e8ea-4cba-b083-4cf864b8186d tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd084bb37aa2544ebb541ddf5b73fe892', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a15d7d5005b49e7b86057e09ca2883f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:38.157144 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21788a45-e8ea-4cba-b083-4cf864b8186d tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:38.157557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21788a45-e8ea-4cba-b083-4cf864b8186d tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:38.157855 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21788a45-e8ea-4cba-b083-4cf864b8186d tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:38.159079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21788a45-e8ea-4cba-b083-4cf864b8186d tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:38.161298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21788a45-e8ea-4cba-b083-4cf864b8186d tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:38.289125 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 999/1959] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:38 2022] POST /identity/v3/projects => generated 353 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:38.291191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6772dd5-6d76-482a-bcd1-80aaa12b01fc tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:38.310881 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6772dd5-6d76-482a-bcd1-80aaa12b01fc tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:38.403772 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3ca30-24ff-43fb-b1d0-c0f1604a5457 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd084bb37aa2544ebb541ddf5b73fe892', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a15d7d5005b49e7b86057e09ca2883f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:38.404462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3ca30-24ff-43fb-b1d0-c0f1604a5457 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:38.404789 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3ca30-24ff-43fb-b1d0-c0f1604a5457 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:38.405077 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3ca30-24ff-43fb-b1d0-c0f1604a5457 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:38.405824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f3ca30-24ff-43fb-b1d0-c0f1604a5457 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:46:38.406139 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f3ca30-24ff-43fb-b1d0-c0f1604a5457 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:38.407861 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f3ca30-24ff-43fb-b1d0-c0f1604a5457 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:38.426829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 961/1960] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:46:38.469876 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21788a45-e8ea-4cba-b083-4cf864b8186d tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:38.482316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21788a45-e8ea-4cba-b083-4cf864b8186d tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:38.488332 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d40553-88f9-4196-95b3-e3757c0ed7e6 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd084bb37aa2544ebb541ddf5b73fe892', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a15d7d5005b49e7b86057e09ca2883f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:38.495031 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d40553-88f9-4196-95b3-e3757c0ed7e6 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:38.495031 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d40553-88f9-4196-95b3-e3757c0ed7e6 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:38.495031 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d40553-88f9-4196-95b3-e3757c0ed7e6 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] PATH_INFO: `/v3/projects/af544cdb58534d64bd317e7f64c70ac9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:38.495031 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d40553-88f9-4196-95b3-e3757c0ed7e6 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] RBAC: Authorizing `identity:delete_project(project_id=af544cdb58534d64bd317e7f64c70ac9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:38.495612 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d40553-88f9-4196-95b3-e3757c0ed7e6 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:38.585299 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3ca30-24ff-43fb-b1d0-c0f1604a5457 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:38.587842 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-89f3ca30-24ff-43fb-b1d0-c0f1604a5457 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:38.602205 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:38.641543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e8d40553-88f9-4196-95b3-e3757c0ed7e6 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af544cdb58534d64bd317e7f64c70ac9', 'request_id': 'req-e8d40553-88f9-4196-95b3-e3757c0ed7e6'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:38.651052 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e8d40553-88f9-4196-95b3-e3757c0ed7e6 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af544cdb58534d64bd317e7f64c70ac9', 'request_id': 'req-e8d40553-88f9-4196-95b3-e3757c0ed7e6'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:38.666939 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e8d40553-88f9-4196-95b3-e3757c0ed7e6 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af544cdb58534d64bd317e7f64c70ac9', 'request_id': 'req-e8d40553-88f9-4196-95b3-e3757c0ed7e6'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:38.713632 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:38.747560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3ca30-24ff-43fb-b1d0-c0f1604a5457 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:38.789205 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e8d40553-88f9-4196-95b3-e3757c0ed7e6 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] The token cache is being invalidate because project af544cdb58534d64bd317e7f64c70ac9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:38.789879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e8d40553-88f9-4196-95b3-e3757c0ed7e6 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:38.791045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e8d40553-88f9-4196-95b3-e3757c0ed7e6 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:38.799479 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1000/1961] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:38 2022] DELETE /identity/v3/projects/af544cdb58534d64bd317e7f64c70ac9 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:38.800610 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d40553-88f9-4196-95b3-e3757c0ed7e6 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:38.803686 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e8d40553-88f9-4196-95b3-e3757c0ed7e6 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:38.836030 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d692924-9c59-4645-be12-eba66a10dce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:38.836755 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d692924-9c59-4645-be12-eba66a10dce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:38.837152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d692924-9c59-4645-be12-eba66a10dce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:38.837541 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d692924-9c59-4645-be12-eba66a10dce6 admin admin] PATH_INFO: `/v3/projects/7ac265d1ab9c42308bbf5441e16597eb` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:38.839012 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d692924-9c59-4645-be12-eba66a10dce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ac265d1ab9c42308bbf5441e16597eb)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:38.841254 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d692924-9c59-4645-be12-eba66a10dce6 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:38.950686 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d40553-88f9-4196-95b3-e3757c0ed7e6 tempest-VolumeQuotaClassesTest-1614239003 tempest-VolumeQuotaClassesTest-1614239003-project-admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:38.969930 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913b902-263e-45c9-a092-bba78d6bcd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:38.970954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913b902-263e-45c9-a092-bba78d6bcd68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:38.971509 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913b902-263e-45c9-a092-bba78d6bcd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:38.971509 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913b902-263e-45c9-a092-bba78d6bcd68 admin admin] PATH_INFO: `/v3/users/2778f510580242bfa31f65892569bf2b` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:38.973005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913b902-263e-45c9-a092-bba78d6bcd68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2778f510580242bfa31f65892569bf2b)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:38.976517 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913b902-263e-45c9-a092-bba78d6bcd68 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:38.999725 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-2d692924-9c59-4645-be12-eba66a10dce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ac265d1ab9c42308bbf5441e16597eb', 'request_id': 'req-2d692924-9c59-4645-be12-eba66a10dce6'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:39.099736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-2d692924-9c59-4645-be12-eba66a10dce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ac265d1ab9c42308bbf5441e16597eb', 'request_id': 'req-2d692924-9c59-4645-be12-eba66a10dce6'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:39.108884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-2d692924-9c59-4645-be12-eba66a10dce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ac265d1ab9c42308bbf5441e16597eb', 'request_id': 'req-2d692924-9c59-4645-be12-eba66a10dce6'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:39.144723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-2d692924-9c59-4645-be12-eba66a10dce6 admin admin] The token cache is being invalidate because project 7ac265d1ab9c42308bbf5441e16597eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:39.145120 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-2d692924-9c59-4645-be12-eba66a10dce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:39.146136 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-2d692924-9c59-4645-be12-eba66a10dce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:39.151492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 962/1962] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:38 2022] DELETE /identity/v3/projects/7ac265d1ab9c42308bbf5441e16597eb => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:39.153295 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d692924-9c59-4645-be12-eba66a10dce6 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:39.156504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-2d692924-9c59-4645-be12-eba66a10dce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:39.211632 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-1913b902-263e-45c9-a092-bba78d6bcd68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2778f510580242bfa31f65892569bf2b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:39.244973 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-1913b902-263e-45c9-a092-bba78d6bcd68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2778f510580242bfa31f65892569bf2b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:39.247164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-1913b902-263e-45c9-a092-bba78d6bcd68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2778f510580242bfa31f65892569bf2b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:39.248670 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-1913b902-263e-45c9-a092-bba78d6bcd68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2778f510580242bfa31f65892569bf2b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:39.278196 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-1913b902-263e-45c9-a092-bba78d6bcd68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2778f510580242bfa31f65892569bf2b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:39.320852 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d692924-9c59-4645-be12-eba66a10dce6 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:39.333213 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e73bd-00d9-4e01-a790-3e17632843cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:39.334852 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e73bd-00d9-4e01-a790-3e17632843cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:39.334852 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e73bd-00d9-4e01-a790-3e17632843cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:39.335160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e73bd-00d9-4e01-a790-3e17632843cc admin admin] PATH_INFO: `/v3/users/8c5d929ec1004638ae06b8aaf6cd55ef` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:39.337490 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e73bd-00d9-4e01-a790-3e17632843cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c5d929ec1004638ae06b8aaf6cd55ef)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:39.352064 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e73bd-00d9-4e01-a790-3e17632843cc admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:39.384219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1001/1963] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:38 2022] DELETE /identity/v3/users/2778f510580242bfa31f65892569bf2b => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:39.386343 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4852b-42c7-4802-a04f-1cd9d40db312 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:39.386866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4852b-42c7-4802-a04f-1cd9d40db312 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:39.388503 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4852b-42c7-4802-a04f-1cd9d40db312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:39.424935 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-38d4852b-42c7-4802-a04f-1cd9d40db312 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:39.426287 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-38d4852b-42c7-4802-a04f-1cd9d40db312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:39.459601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:39.476931 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:39.516778 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1002/1964] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:39.518020 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4852b-42c7-4802-a04f-1cd9d40db312 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:39.519826 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-38d4852b-42c7-4802-a04f-1cd9d40db312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:39.533530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4852b-42c7-4802-a04f-1cd9d40db312 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:39.538305 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32770279-9120-42b4-ae06-b19cf8add47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:39.539204 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32770279-9120-42b4-ae06-b19cf8add47f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:39.539482 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32770279-9120-42b4-ae06-b19cf8add47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:39.540552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32770279-9120-42b4-ae06-b19cf8add47f admin admin] PATH_INFO: `/v3/users/d82614c2d5d64e9ab815b86daa0fd105` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:39.542802 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32770279-9120-42b4-ae06-b19cf8add47f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d82614c2d5d64e9ab815b86daa0fd105)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:39.544788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32770279-9120-42b4-ae06-b19cf8add47f admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:39.598991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-922e73bd-00d9-4e01-a790-3e17632843cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c5d929ec1004638ae06b8aaf6cd55ef'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:39.623424 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-922e73bd-00d9-4e01-a790-3e17632843cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c5d929ec1004638ae06b8aaf6cd55ef'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:39.709151 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-922e73bd-00d9-4e01-a790-3e17632843cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c5d929ec1004638ae06b8aaf6cd55ef'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:39.709975 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-922e73bd-00d9-4e01-a790-3e17632843cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c5d929ec1004638ae06b8aaf6cd55ef'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:39.713305 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-922e73bd-00d9-4e01-a790-3e17632843cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c5d929ec1004638ae06b8aaf6cd55ef'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:39.746940 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 963/1965] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:39 2022] DELETE /identity/v3/users/8c5d929ec1004638ae06b8aaf6cd55ef => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:39.748395 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e73bd-00d9-4e01-a790-3e17632843cc admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:39.750850 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-922e73bd-00d9-4e01-a790-3e17632843cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:39.843075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e73bd-00d9-4e01-a790-3e17632843cc admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:39.861796 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-32770279-9120-42b4-ae06-b19cf8add47f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd82614c2d5d64e9ab815b86daa0fd105'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:39.875901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9293b-8829-4865-a136-bde8c8dd5bef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:39.876597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9293b-8829-4865-a136-bde8c8dd5bef service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:39.879184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9293b-8829-4865-a136-bde8c8dd5bef service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:39.879490 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9293b-8829-4865-a136-bde8c8dd5bef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:39.886660 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-32e9293b-8829-4865-a136-bde8c8dd5bef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:39.889425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-32770279-9120-42b4-ae06-b19cf8add47f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd82614c2d5d64e9ab815b86daa0fd105'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:39.891144 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-32770279-9120-42b4-ae06-b19cf8add47f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd82614c2d5d64e9ab815b86daa0fd105'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:39.892078 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-32770279-9120-42b4-ae06-b19cf8add47f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd82614c2d5d64e9ab815b86daa0fd105'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:39.908382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-32770279-9120-42b4-ae06-b19cf8add47f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd82614c2d5d64e9ab815b86daa0fd105'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:39.996315 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1003/1966] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:39 2022] DELETE /identity/v3/users/d82614c2d5d64e9ab815b86daa0fd105 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:39.999495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ad78e-1df1-43e4-91be-8a9efbd996e5 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:39.999917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ad78e-1df1-43e4-91be-8a9efbd996e5 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:40.000490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ad78e-1df1-43e4-91be-8a9efbd996e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:40.033586 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9293b-8829-4865-a136-bde8c8dd5bef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:40.035592 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9293b-8829-4865-a136-bde8c8dd5bef service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:40.042656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-169ad78e-1df1-43e4-91be-8a9efbd996e5 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:40.048279 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-169ad78e-1df1-43e4-91be-8a9efbd996e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:40.048747 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 964/1967] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:40.185614 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1004/1968] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:46:40.210658 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9293b-8829-4865-a136-bde8c8dd5bef service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:40.221721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9293b-8829-4865-a136-bde8c8dd5bef service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:40.341338 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6605d-693a-4660-b073-04bee7c2efe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:40.343138 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6605d-693a-4660-b073-04bee7c2efe5 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:40.344171 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6605d-693a-4660-b073-04bee7c2efe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:40.344647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6605d-693a-4660-b073-04bee7c2efe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:40.367266 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6605d-693a-4660-b073-04bee7c2efe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:40.367266 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6605d-693a-4660-b073-04bee7c2efe5 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:40.372842 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:40.381355 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 965/1969] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:40.637165 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ad78e-1df1-43e4-91be-8a9efbd996e5 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:40.638435 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-169ad78e-1df1-43e4-91be-8a9efbd996e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:40.658767 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ad78e-1df1-43e4-91be-8a9efbd996e5 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:40.672994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd37b19-d316-4f0f-bf1f-35ee3aed801d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:40.674143 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd37b19-d316-4f0f-bf1f-35ee3aed801d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:40.674582 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd37b19-d316-4f0f-bf1f-35ee3aed801d admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:40.675011 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd37b19-d316-4f0f-bf1f-35ee3aed801d admin admin] PATH_INFO: `/v3/projects/5a8ff6a49e69456f9a6c28bb157c79fc` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:40.676457 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd37b19-d316-4f0f-bf1f-35ee3aed801d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a8ff6a49e69456f9a6c28bb157c79fc)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:40.679312 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd37b19-d316-4f0f-bf1f-35ee3aed801d admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:40.875255 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:40.878187 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3fd37b19-d316-4f0f-bf1f-35ee3aed801d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a8ff6a49e69456f9a6c28bb157c79fc', 'request_id': 'req-3fd37b19-d316-4f0f-bf1f-35ee3aed801d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:40.887576 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3fd37b19-d316-4f0f-bf1f-35ee3aed801d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a8ff6a49e69456f9a6c28bb157c79fc', 'request_id': 'req-3fd37b19-d316-4f0f-bf1f-35ee3aed801d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:40.902784 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3fd37b19-d316-4f0f-bf1f-35ee3aed801d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a8ff6a49e69456f9a6c28bb157c79fc', 'request_id': 'req-3fd37b19-d316-4f0f-bf1f-35ee3aed801d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:40.958209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6605d-693a-4660-b073-04bee7c2efe5 service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:40.959895 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-6fd6605d-693a-4660-b073-04bee7c2efe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:40.982193 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6605d-693a-4660-b073-04bee7c2efe5 service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:41.013773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3fd37b19-d316-4f0f-bf1f-35ee3aed801d admin admin] The token cache is being invalidate because project 5a8ff6a49e69456f9a6c28bb157c79fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:41.014771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3fd37b19-d316-4f0f-bf1f-35ee3aed801d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.015230 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3fd37b19-d316-4f0f-bf1f-35ee3aed801d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.018271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1005/1970] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:40 2022] DELETE /identity/v3/projects/5a8ff6a49e69456f9a6c28bb157c79fc => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:41.019166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd37b19-d316-4f0f-bf1f-35ee3aed801d admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:41.021402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3fd37b19-d316-4f0f-bf1f-35ee3aed801d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:41.112429 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91775dc-8bb7-4e8c-8c8e-4b00a0c24420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:41.114180 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91775dc-8bb7-4e8c-8c8e-4b00a0c24420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:41.114535 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91775dc-8bb7-4e8c-8c8e-4b00a0c24420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:41.115045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91775dc-8bb7-4e8c-8c8e-4b00a0c24420 admin admin] PATH_INFO: `/v3/projects/87b6152d7e574439a354f020eb1fcfdd` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:41.116992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91775dc-8bb7-4e8c-8c8e-4b00a0c24420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87b6152d7e574439a354f020eb1fcfdd)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:41.119703 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91775dc-8bb7-4e8c-8c8e-4b00a0c24420 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:41.135976 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd37b19-d316-4f0f-bf1f-35ee3aed801d admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:41.140691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d8238-e232-4e71-ad2f-93b7c49c6e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:41.143321 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d8238-e232-4e71-ad2f-93b7c49c6e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:41.143778 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d8238-e232-4e71-ad2f-93b7c49c6e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:41.144205 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d8238-e232-4e71-ad2f-93b7c49c6e19 admin admin] PATH_INFO: `/v3/projects/825b91d2627543fbb7ce3c5a4f77b7de` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:41.147506 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d8238-e232-4e71-ad2f-93b7c49c6e19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=825b91d2627543fbb7ce3c5a4f77b7de)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:41.151227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d8238-e232-4e71-ad2f-93b7c49c6e19 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:41.298552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-f14d8238-e232-4e71-ad2f-93b7c49c6e19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '825b91d2627543fbb7ce3c5a4f77b7de', 'request_id': 'req-f14d8238-e232-4e71-ad2f-93b7c49c6e19'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.305339 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-c91775dc-8bb7-4e8c-8c8e-4b00a0c24420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87b6152d7e574439a354f020eb1fcfdd', 'request_id': 'req-c91775dc-8bb7-4e8c-8c8e-4b00a0c24420'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.308359 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-f14d8238-e232-4e71-ad2f-93b7c49c6e19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '825b91d2627543fbb7ce3c5a4f77b7de', 'request_id': 'req-f14d8238-e232-4e71-ad2f-93b7c49c6e19'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.318289 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-f14d8238-e232-4e71-ad2f-93b7c49c6e19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '825b91d2627543fbb7ce3c5a4f77b7de', 'request_id': 'req-f14d8238-e232-4e71-ad2f-93b7c49c6e19'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.384372 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-c91775dc-8bb7-4e8c-8c8e-4b00a0c24420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87b6152d7e574439a354f020eb1fcfdd', 'request_id': 'req-c91775dc-8bb7-4e8c-8c8e-4b00a0c24420'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.395981 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-c91775dc-8bb7-4e8c-8c8e-4b00a0c24420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87b6152d7e574439a354f020eb1fcfdd', 'request_id': 'req-c91775dc-8bb7-4e8c-8c8e-4b00a0c24420'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.410207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-c91775dc-8bb7-4e8c-8c8e-4b00a0c24420 admin admin] The token cache is being invalidate because project 87b6152d7e574439a354f020eb1fcfdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:41.410595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-c91775dc-8bb7-4e8c-8c8e-4b00a0c24420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.411412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-c91775dc-8bb7-4e8c-8c8e-4b00a0c24420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.413386 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 966/1971] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:40 2022] DELETE /identity/v3/projects/87b6152d7e574439a354f020eb1fcfdd => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:41.414931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c1cbd-7533-4c0c-8578-3ca419311488 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:41.415531 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c1cbd-7533-4c0c-8578-3ca419311488 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:41.416028 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c1cbd-7533-4c0c-8578-3ca419311488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:41.441692 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-f14d8238-e232-4e71-ad2f-93b7c49c6e19 admin admin] The token cache is being invalidate because project 825b91d2627543fbb7ce3c5a4f77b7de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:41.442113 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-f14d8238-e232-4e71-ad2f-93b7c49c6e19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.446419 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-f14d8238-e232-4e71-ad2f-93b7c49c6e19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.446419 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1006/1972] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:41 2022] DELETE /identity/v3/projects/825b91d2627543fbb7ce3c5a4f77b7de => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:41.447799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d8238-e232-4e71-ad2f-93b7c49c6e19 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:41.450852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-f14d8238-e232-4e71-ad2f-93b7c49c6e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:41.455650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-ac9c1cbd-7533-4c0c-8578-3ca419311488 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:41.457495 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ac9c1cbd-7533-4c0c-8578-3ca419311488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:41.584591 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d8238-e232-4e71-ad2f-93b7c49c6e19 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:41.600648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 967/1973] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:41.601410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c1cbd-7533-4c0c-8578-3ca419311488 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:41.603883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ac9c1cbd-7533-4c0c-8578-3ca419311488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:41.623086 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c1cbd-7533-4c0c-8578-3ca419311488 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:41.625889 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26081ce-c506-44c1-9132-a6f526619d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:41.627510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26081ce-c506-44c1-9132-a6f526619d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:41.627835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26081ce-c506-44c1-9132-a6f526619d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:41.628124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26081ce-c506-44c1-9132-a6f526619d91 admin admin] PATH_INFO: `/v3/users/d084bb37aa2544ebb541ddf5b73fe892` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:41.631356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26081ce-c506-44c1-9132-a6f526619d91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d084bb37aa2544ebb541ddf5b73fe892)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:41.633414 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26081ce-c506-44c1-9132-a6f526619d91 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:41.681737 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f3067-4edb-4a15-bfab-232118521cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:41.686289 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f3067-4edb-4a15-bfab-232118521cd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:41.686289 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f3067-4edb-4a15-bfab-232118521cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:41.686289 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f3067-4edb-4a15-bfab-232118521cd9 admin admin] PATH_INFO: `/v3/users/cae23cde84d04abbbefdbfd5bfe9eb94` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:41.686289 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f3067-4edb-4a15-bfab-232118521cd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cae23cde84d04abbbefdbfd5bfe9eb94)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:41.687382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f3067-4edb-4a15-bfab-232118521cd9 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:41.878230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d26081ce-c506-44c1-9132-a6f526619d91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd084bb37aa2544ebb541ddf5b73fe892'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.895024 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d26081ce-c506-44c1-9132-a6f526619d91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd084bb37aa2544ebb541ddf5b73fe892'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.926147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:41.932179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-9a9f3067-4edb-4a15-bfab-232118521cd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cae23cde84d04abbbefdbfd5bfe9eb94'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.949844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-9a9f3067-4edb-4a15-bfab-232118521cd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cae23cde84d04abbbefdbfd5bfe9eb94'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.950638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-9a9f3067-4edb-4a15-bfab-232118521cd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cae23cde84d04abbbefdbfd5bfe9eb94'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.951255 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-9a9f3067-4edb-4a15-bfab-232118521cd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cae23cde84d04abbbefdbfd5bfe9eb94'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.959978 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d26081ce-c506-44c1-9132-a6f526619d91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd084bb37aa2544ebb541ddf5b73fe892'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.960633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d26081ce-c506-44c1-9132-a6f526619d91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd084bb37aa2544ebb541ddf5b73fe892'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.961302 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-9a9f3067-4edb-4a15-bfab-232118521cd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cae23cde84d04abbbefdbfd5bfe9eb94'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.961697 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d26081ce-c506-44c1-9132-a6f526619d91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd084bb37aa2544ebb541ddf5b73fe892'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:41.982757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 968/1974] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:41 2022] DELETE /identity/v3/users/d084bb37aa2544ebb541ddf5b73fe892 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:41.984149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f90ca-8f26-4c98-9f7a-b06381039adf None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:41.984445 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f90ca-8f26-4c98-9f7a-b06381039adf None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:41.985400 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f90ca-8f26-4c98-9f7a-b06381039adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:42.007471 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-044f90ca-8f26-4c98-9f7a-b06381039adf None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:42.010491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-044f90ca-8f26-4c98-9f7a-b06381039adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:42.046516 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1007/1975] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:41 2022] DELETE /identity/v3/users/cae23cde84d04abbbefdbfd5bfe9eb94 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:42.051191 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbb196-4b0e-4b11-93e7-314637529576 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:42.051191 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbb196-4b0e-4b11-93e7-314637529576 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:42.051191 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbb196-4b0e-4b11-93e7-314637529576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:42.096326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-cdcbb196-4b0e-4b11-93e7-314637529576 None None] MFA Rules not processed for user `e2e0900058f7442fa48c48903781317c`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:42.097957 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-cdcbb196-4b0e-4b11-93e7-314637529576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:42.119692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 969/1976] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:42.124700 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f90ca-8f26-4c98-9f7a-b06381039adf None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:42.136709 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f90ca-8f26-4c98-9f7a-b06381039adf None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:42.188216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1008/1977] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:42 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:42.203340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbb196-4b0e-4b11-93e7-314637529576 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:42.205272 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-cdcbb196-4b0e-4b11-93e7-314637529576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:42.213137 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b6297-7efb-4c41-a225-05af1f6e167a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:42.214132 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b6297-7efb-4c41-a225-05af1f6e167a admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:42.214512 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b6297-7efb-4c41-a225-05af1f6e167a admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:42.214797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b6297-7efb-4c41-a225-05af1f6e167a admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:42.215759 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b6297-7efb-4c41-a225-05af1f6e167a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:46:42.216255 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b6297-7efb-4c41-a225-05af1f6e167a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:42.218954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b6297-7efb-4c41-a225-05af1f6e167a admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:42.239146 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 970/1978] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:46:42.248182 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b6297-7efb-4c41-a225-05af1f6e167a admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:42.264115 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b6297-7efb-4c41-a225-05af1f6e167a admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:42.277262 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52052e84-e4cc-431f-b987-249a7d45cfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:42.277953 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52052e84-e4cc-431f-b987-249a7d45cfcf admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:42.279046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52052e84-e4cc-431f-b987-249a7d45cfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:42.279344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52052e84-e4cc-431f-b987-249a7d45cfcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:42.284911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52052e84-e4cc-431f-b987-249a7d45cfcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:42.286995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52052e84-e4cc-431f-b987-249a7d45cfcf admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:42.322964 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:42.341415 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbb196-4b0e-4b11-93e7-314637529576 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:42.348032 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e037ca7-ccf0-4a92-9084-efa1cd51f3bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:42.349202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e037ca7-ccf0-4a92-9084-efa1cd51f3bb service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:42.349202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e037ca7-ccf0-4a92-9084-efa1cd51f3bb service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:42.354054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e037ca7-ccf0-4a92-9084-efa1cd51f3bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:42.354054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6e037ca7-ccf0-4a92-9084-efa1cd51f3bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:42.376672 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 971/1979] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:42 2022] POST /identity/v3/projects => generated 401 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:42.380792 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52052e84-e4cc-431f-b987-249a7d45cfcf admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:42.444996 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e037ca7-ccf0-4a92-9084-efa1cd51f3bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:42.447768 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e037ca7-ccf0-4a92-9084-efa1cd51f3bb service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:42.462219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1009/1980] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:42.481425 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52052e84-e4cc-431f-b987-249a7d45cfcf admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:42.488635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bdc88-f894-47ec-a5c3-1d57334e9631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:42.489386 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bdc88-f894-47ec-a5c3-1d57334e9631 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:42.489752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bdc88-f894-47ec-a5c3-1d57334e9631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:42.490173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bdc88-f894-47ec-a5c3-1d57334e9631 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:42.491022 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43bdc88-f894-47ec-a5c3-1d57334e9631 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:42.493484 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43bdc88-f894-47ec-a5c3-1d57334e9631 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:42.556648 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e037ca7-ccf0-4a92-9084-efa1cd51f3bb service nova] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:42.558284 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6e037ca7-ccf0-4a92-9084-efa1cd51f3bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:42.576368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e037ca7-ccf0-4a92-9084-efa1cd51f3bb service nova] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:42.593948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715336e-bdd0-4b61-8973-d7b909d67fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:42.594846 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715336e-bdd0-4b61-8973-d7b909d67fd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:42.595133 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715336e-bdd0-4b61-8973-d7b909d67fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:42.595407 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715336e-bdd0-4b61-8973-d7b909d67fd7 admin admin] PATH_INFO: `/v3/projects/8a15d7d5005b49e7b86057e09ca2883f` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:42.595524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 972/1981] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:42 2022] POST /identity/v3/users => generated 301 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:46:42.596661 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715336e-bdd0-4b61-8973-d7b909d67fd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a15d7d5005b49e7b86057e09ca2883f)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:42.598894 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bdc88-f894-47ec-a5c3-1d57334e9631 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:42.601513 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715336e-bdd0-4b61-8973-d7b909d67fd7 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:42.622076 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bdc88-f894-47ec-a5c3-1d57334e9631 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:42.629805 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eabc18-e2d8-4463-86dd-349ec2a477fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:42.630859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eabc18-e2d8-4463-86dd-349ec2a477fa admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:42.631116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eabc18-e2d8-4463-86dd-349ec2a477fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:42.631459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eabc18-e2d8-4463-86dd-349ec2a477fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:42.632291 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eabc18-e2d8-4463-86dd-349ec2a477fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:42.634322 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eabc18-e2d8-4463-86dd-349ec2a477fa admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:42.648608 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 973/1982] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:42.653124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eabc18-e2d8-4463-86dd-349ec2a477fa admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:42.663924 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eabc18-e2d8-4463-86dd-349ec2a477fa admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:42.678277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f013de6e-ac9f-49cc-964e-d9244b48e50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:42.679605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013de6e-ac9f-49cc-964e-d9244b48e50e admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:42.680366 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013de6e-ac9f-49cc-964e-d9244b48e50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:42.680687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013de6e-ac9f-49cc-964e-d9244b48e50e admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:42.682981 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f013de6e-ac9f-49cc-964e-d9244b48e50e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:42.683835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f013de6e-ac9f-49cc-964e-d9244b48e50e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:42.702879 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 974/1983] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:42.703992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f013de6e-ac9f-49cc-964e-d9244b48e50e admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:42.709273 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-f013de6e-ac9f-49cc-964e-d9244b48e50e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:42.728465 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f013de6e-ac9f-49cc-964e-d9244b48e50e admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:42.734349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e85c3a-761a-45bf-9ec7-f835ac68fab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:42.735345 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e85c3a-761a-45bf-9ec7-f835ac68fab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:42.735635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e85c3a-761a-45bf-9ec7-f835ac68fab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:42.736117 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e85c3a-761a-45bf-9ec7-f835ac68fab8 admin admin] PATH_INFO: `/v3/projects/d6f9d6c8fcc3442db15483575c444739` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:42.738377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e85c3a-761a-45bf-9ec7-f835ac68fab8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6f9d6c8fcc3442db15483575c444739)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:42.740147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e85c3a-761a-45bf-9ec7-f835ac68fab8 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:42.782830 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-4715336e-bdd0-4b61-8973-d7b909d67fd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a15d7d5005b49e7b86057e09ca2883f', 'request_id': 'req-4715336e-bdd0-4b61-8973-d7b909d67fd7'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:42.791389 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-4715336e-bdd0-4b61-8973-d7b909d67fd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a15d7d5005b49e7b86057e09ca2883f', 'request_id': 'req-4715336e-bdd0-4b61-8973-d7b909d67fd7'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:42.805427 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-4715336e-bdd0-4b61-8973-d7b909d67fd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a15d7d5005b49e7b86057e09ca2883f', 'request_id': 'req-4715336e-bdd0-4b61-8973-d7b909d67fd7'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:42.825979 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:42.871292 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d2e85c3a-761a-45bf-9ec7-f835ac68fab8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6f9d6c8fcc3442db15483575c444739', 'request_id': 'req-d2e85c3a-761a-45bf-9ec7-f835ac68fab8'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:42.890241 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-4715336e-bdd0-4b61-8973-d7b909d67fd7 admin admin] The token cache is being invalidate because project 8a15d7d5005b49e7b86057e09ca2883f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:42.890628 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-4715336e-bdd0-4b61-8973-d7b909d67fd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:42.891903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-4715336e-bdd0-4b61-8973-d7b909d67fd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:42.894679 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1010/1984] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:42 2022] DELETE /identity/v3/projects/8a15d7d5005b49e7b86057e09ca2883f => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:42.896248 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715336e-bdd0-4b61-8973-d7b909d67fd7 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:42.897788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-4715336e-bdd0-4b61-8973-d7b909d67fd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:42.971346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d2e85c3a-761a-45bf-9ec7-f835ac68fab8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6f9d6c8fcc3442db15483575c444739', 'request_id': 'req-d2e85c3a-761a-45bf-9ec7-f835ac68fab8'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:42.978442 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d2e85c3a-761a-45bf-9ec7-f835ac68fab8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6f9d6c8fcc3442db15483575c444739', 'request_id': 'req-d2e85c3a-761a-45bf-9ec7-f835ac68fab8'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:42.979864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715336e-bdd0-4b61-8973-d7b909d67fd7 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:42.989228 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d2e85c3a-761a-45bf-9ec7-f835ac68fab8 admin admin] The token cache is being invalidate because project d6f9d6c8fcc3442db15483575c444739 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:42.989584 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d2e85c3a-761a-45bf-9ec7-f835ac68fab8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:42.990187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d2e85c3a-761a-45bf-9ec7-f835ac68fab8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:42.991764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 975/1985] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:42 2022] DELETE /identity/v3/projects/d6f9d6c8fcc3442db15483575c444739 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:42.993226 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebc1af-a1b4-479e-b9e3-44bdbe921689 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:42.993434 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebc1af-a1b4-479e-b9e3-44bdbe921689 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:42.993709 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebc1af-a1b4-479e-b9e3-44bdbe921689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:43.014507 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-81ebc1af-a1b4-479e-b9e3-44bdbe921689 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:43.016050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-81ebc1af-a1b4-479e-b9e3-44bdbe921689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:43.052401 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f7b67-f770-4d80-a6e9-ebb234737349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:43.053489 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f7b67-f770-4d80-a6e9-ebb234737349 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:43.053899 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f7b67-f770-4d80-a6e9-ebb234737349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:43.054183 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f7b67-f770-4d80-a6e9-ebb234737349 admin admin] PATH_INFO: `/v3/projects/0a246906083b4b52a9585d852e14458e/users/42e087686e8d4af48140c53bb583add3/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:43.080884 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f7b67-f770-4d80-a6e9-ebb234737349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a246906083b4b52a9585d852e14458e, user_id=42e087686e8d4af48140c53bb583add3, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:43.091600 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 976/1986] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:43.091891 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f7b67-f770-4d80-a6e9-ebb234737349 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:43.091971 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cc82b-f130-42fd-9060-f1d7a892f695 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:43.091971 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cc82b-f130-42fd-9060-f1d7a892f695 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:43.091971 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cc82b-f130-42fd-9060-f1d7a892f695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:43.106385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-d49cc82b-f130-42fd-9060-f1d7a892f695 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:43.108067 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-d49cc82b-f130-42fd-9060-f1d7a892f695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:43.116938 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 977/1987] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:43.118599 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791c54e-c645-4943-8929-f3b9e4b3d6a1 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:43.118891 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791c54e-c645-4943-8929-f3b9e4b3d6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:43.119167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791c54e-c645-4943-8929-f3b9e4b3d6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:43.139469 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-7791c54e-c645-4943-8929-f3b9e4b3d6a1 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:43.147998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-7791c54e-c645-4943-8929-f3b9e4b3d6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:43.151758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 978/1988] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:43.159149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261e3f4-96e8-4d44-8837-8f34488526f2 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:43.159504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261e3f4-96e8-4d44-8837-8f34488526f2 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:43.159843 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261e3f4-96e8-4d44-8837-8f34488526f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:43.185853 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-2261e3f4-96e8-4d44-8837-8f34488526f2 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:43.187319 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-2261e3f4-96e8-4d44-8837-8f34488526f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:43.195558 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 979/1989] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:43.197185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261e3f4-96e8-4d44-8837-8f34488526f2 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:43.207376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1011/1990] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:42 2022] PUT /identity/v3/projects/0a246906083b4b52a9585d852e14458e/users/42e087686e8d4af48140c53bb583add3/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:43.207912 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261e3f4-96e8-4d44-8837-8f34488526f2 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:43.208633 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f7b67-f770-4d80-a6e9-ebb234737349 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:43.276847 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459e20d-23df-484b-a2e5-d6c1f879ed2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:43.277758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459e20d-23df-484b-a2e5-d6c1f879ed2b admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:43.278046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459e20d-23df-484b-a2e5-d6c1f879ed2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:43.278322 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459e20d-23df-484b-a2e5-d6c1f879ed2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:43.279123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459e20d-23df-484b-a2e5-d6c1f879ed2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:46:43.279283 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f7b67-f770-4d80-a6e9-ebb234737349 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:43.279637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459e20d-23df-484b-a2e5-d6c1f879ed2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:43.281239 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459e20d-23df-484b-a2e5-d6c1f879ed2b admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:43.284206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce88d7d-21aa-41c0-be58-0134d1b40f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:43.284906 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce88d7d-21aa-41c0-be58-0134d1b40f1f admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:43.285188 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce88d7d-21aa-41c0-be58-0134d1b40f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:43.285463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce88d7d-21aa-41c0-be58-0134d1b40f1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:43.286200 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce88d7d-21aa-41c0-be58-0134d1b40f1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:46:43.286507 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce88d7d-21aa-41c0-be58-0134d1b40f1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:43.288097 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce88d7d-21aa-41c0-be58-0134d1b40f1f admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:43.298510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 980/1991] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:46:43.298510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9af1d7-d645-45e7-a812-1f1201ed31fa None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:43.298510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9af1d7-d645-45e7-a812-1f1201ed31fa None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:43.298510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9af1d7-d645-45e7-a812-1f1201ed31fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:43.306108 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1012/1992] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:46:43.307154 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce88d7d-21aa-41c0-be58-0134d1b40f1f admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:43.320474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce88d7d-21aa-41c0-be58-0134d1b40f1f admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:43.327091 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-be9af1d7-d645-45e7-a812-1f1201ed31fa None None] MFA Rules not processed for user `42e087686e8d4af48140c53bb583add3`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:43.327914 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad16630-0968-40f2-b182-15cadd801f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:43.328550 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-be9af1d7-d645-45e7-a812-1f1201ed31fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:43.328690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad16630-0968-40f2-b182-15cadd801f3b admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:43.329335 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad16630-0968-40f2-b182-15cadd801f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:43.329335 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad16630-0968-40f2-b182-15cadd801f3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:43.329963 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad16630-0968-40f2-b182-15cadd801f3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:43.332357 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad16630-0968-40f2-b182-15cadd801f3b admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:43.399797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 981/1993] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:43.400833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9af1d7-d645-45e7-a812-1f1201ed31fa None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:43.410633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9af1d7-d645-45e7-a812-1f1201ed31fa None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:43.415379 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57201557-e156-4624-941f-3590d428b889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:43.416042 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57201557-e156-4624-941f-3590d428b889 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:43.416324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57201557-e156-4624-941f-3590d428b889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:43.416630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57201557-e156-4624-941f-3590d428b889 admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:43.417320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57201557-e156-4624-941f-3590d428b889 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:43.419219 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57201557-e156-4624-941f-3590d428b889 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:43.471045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:43.479557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1013/1994] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:43 2022] POST /identity/v3/projects => generated 387 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:43.480652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad16630-0968-40f2-b182-15cadd801f3b admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:43.482488 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3ad16630-0968-40f2-b182-15cadd801f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:43.533263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 982/1995] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:43 2022] POST /identity/v3/projects => generated 381 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:43.535300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57201557-e156-4624-941f-3590d428b889 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:43.559665 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad16630-0968-40f2-b182-15cadd801f3b admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:43.613325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57201557-e156-4624-941f-3590d428b889 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:43.619414 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8d940-a029-414f-9cc2-ee776f75135c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:43.620101 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8d940-a029-414f-9cc2-ee776f75135c admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:43.620672 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8d940-a029-414f-9cc2-ee776f75135c admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:43.621503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8d940-a029-414f-9cc2-ee776f75135c admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:43.621708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8d940-a029-414f-9cc2-ee776f75135c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:43.623738 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8d940-a029-414f-9cc2-ee776f75135c admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:43.639617 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb94392-7155-42b8-8be9-e354ff51c793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:43.640551 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb94392-7155-42b8-8be9-e354ff51c793 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:43.640868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb94392-7155-42b8-8be9-e354ff51c793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:43.641143 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb94392-7155-42b8-8be9-e354ff51c793 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:43.641837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb94392-7155-42b8-8be9-e354ff51c793 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:43.643731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb94392-7155-42b8-8be9-e354ff51c793 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:43.710460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 983/1996] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:43 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:43.711934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8d940-a029-414f-9cc2-ee776f75135c admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:43.727676 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8d940-a029-414f-9cc2-ee776f75135c admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:43.735461 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6ad61-60cd-4b3c-b17d-08db9f07825f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:43.736848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6ad61-60cd-4b3c-b17d-08db9f07825f admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:43.743598 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6ad61-60cd-4b3c-b17d-08db9f07825f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:43.743598 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6ad61-60cd-4b3c-b17d-08db9f07825f admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:43.743598 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6ad61-60cd-4b3c-b17d-08db9f07825f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:43.743598 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6ad61-60cd-4b3c-b17d-08db9f07825f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:43.762170 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1014/1997] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:43 2022] POST /identity/v3/projects => generated 399 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:43.762854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb94392-7155-42b8-8be9-e354ff51c793 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:43.832737 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 984/1998] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:43 2022] POST /identity/v3/users => generated 291 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:43.832737 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6ad61-60cd-4b3c-b17d-08db9f07825f admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:43.835610 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb94392-7155-42b8-8be9-e354ff51c793 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:43.841837 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6ad61-60cd-4b3c-b17d-08db9f07825f admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:43.842043 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee04ae-9153-4887-a2e6-265c841c0732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:43.842755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee04ae-9153-4887-a2e6-265c841c0732 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:43.843042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee04ae-9153-4887-a2e6-265c841c0732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:43.843324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee04ae-9153-4887-a2e6-265c841c0732 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:43.844175 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee04ae-9153-4887-a2e6-265c841c0732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:43.852603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee04ae-9153-4887-a2e6-265c841c0732 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:43.852761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8201e8-02f5-44e4-bb99-2dddc85c95e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:43.852761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8201e8-02f5-44e4-bb99-2dddc85c95e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:43.852761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8201e8-02f5-44e4-bb99-2dddc85c95e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:43.852761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8201e8-02f5-44e4-bb99-2dddc85c95e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:43.853067 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8201e8-02f5-44e4-bb99-2dddc85c95e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:43.853067 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8201e8-02f5-44e4-bb99-2dddc85c95e9 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:43.858130 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1015/1999] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:43.859172 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee04ae-9153-4887-a2e6-265c841c0732 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:43.869829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee04ae-9153-4887-a2e6-265c841c0732 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:43.875694 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb60fc0-7484-4b85-abdf-b16724605a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:43.876434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb60fc0-7484-4b85-abdf-b16724605a88 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:43.876815 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb60fc0-7484-4b85-abdf-b16724605a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:43.877204 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb60fc0-7484-4b85-abdf-b16724605a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:43.877962 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb60fc0-7484-4b85-abdf-b16724605a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:43.879666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb60fc0-7484-4b85-abdf-b16724605a88 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:43.889870 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1016/2000] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:43.890662 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb60fc0-7484-4b85-abdf-b16724605a88 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:43.900929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb60fc0-7484-4b85-abdf-b16724605a88 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:43.906714 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c441957-58f1-48bd-ae76-3ae2e2c86dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:43.907374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c441957-58f1-48bd-ae76-3ae2e2c86dbc admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:43.907718 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c441957-58f1-48bd-ae76-3ae2e2c86dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:43.908009 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c441957-58f1-48bd-ae76-3ae2e2c86dbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:43.908793 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c441957-58f1-48bd-ae76-3ae2e2c86dbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:43.911982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c441957-58f1-48bd-ae76-3ae2e2c86dbc admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:43.926085 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1017/2001] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:43.927035 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c441957-58f1-48bd-ae76-3ae2e2c86dbc admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:43.945808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c441957-58f1-48bd-ae76-3ae2e2c86dbc admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:43.954566 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca30f4-a58b-4a23-8203-54b94fc1fd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:43.955355 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca30f4-a58b-4a23-8203-54b94fc1fd71 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:43.956066 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca30f4-a58b-4a23-8203-54b94fc1fd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:43.956706 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca30f4-a58b-4a23-8203-54b94fc1fd71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:43.957643 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca30f4-a58b-4a23-8203-54b94fc1fd71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:43.959245 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 985/2002] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:43 2022] POST /identity/v3/users => generated 306 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:43.960264 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8201e8-02f5-44e4-bb99-2dddc85c95e9 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:43.963191 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca30f4-a58b-4a23-8203-54b94fc1fd71 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:43.975686 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1018/2003] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:43.977029 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca30f4-a58b-4a23-8203-54b94fc1fd71 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:43.979098 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8201e8-02f5-44e4-bb99-2dddc85c95e9 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:43.989156 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d555c-1db1-42d4-ad24-bba073ea6098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:43.989408 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca30f4-a58b-4a23-8203-54b94fc1fd71 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:43.990233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d555c-1db1-42d4-ad24-bba073ea6098 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:43.990595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d555c-1db1-42d4-ad24-bba073ea6098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:43.991073 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d555c-1db1-42d4-ad24-bba073ea6098 admin admin] PATH_INFO: `/v3/projects/6e4755fbd98d4b8cb9b40ddfa178c421/users/3f08821872cb4db389110a1efac1d6a7/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:43.995550 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa1630-14e3-49c2-aa1f-f239c514d5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:43.996490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa1630-14e3-49c2-aa1f-f239c514d5b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:43.997236 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa1630-14e3-49c2-aa1f-f239c514d5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:43.997790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa1630-14e3-49c2-aa1f-f239c514d5b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:43.998918 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa1630-14e3-49c2-aa1f-f239c514d5b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:44.000860 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa1630-14e3-49c2-aa1f-f239c514d5b3 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:44.019515 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d555c-1db1-42d4-ad24-bba073ea6098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e4755fbd98d4b8cb9b40ddfa178c421, user_id=3f08821872cb4db389110a1efac1d6a7, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:44.024974 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d555c-1db1-42d4-ad24-bba073ea6098 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:44.029517 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1019/2004] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:44.030658 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa1630-14e3-49c2-aa1f-f239c514d5b3 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:44.041257 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:44.050403 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa1630-14e3-49c2-aa1f-f239c514d5b3 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:44.057247 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d724e0-618d-4c7b-815f-8d2f1a139362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:44.058381 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d724e0-618d-4c7b-815f-8d2f1a139362 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:44.058835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d724e0-618d-4c7b-815f-8d2f1a139362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:44.059375 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d724e0-618d-4c7b-815f-8d2f1a139362 admin admin] PATH_INFO: `/v3/projects/0d30f6fdf7274315ad3566051adddf03/users/eac5cbf36b68469e82a01f263155770c/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:44.082771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d724e0-618d-4c7b-815f-8d2f1a139362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d30f6fdf7274315ad3566051adddf03, user_id=eac5cbf36b68469e82a01f263155770c, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:44.090386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d724e0-618d-4c7b-815f-8d2f1a139362 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:44.118540 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 986/2005] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:43 2022] PUT /identity/v3/projects/6e4755fbd98d4b8cb9b40ddfa178c421/users/3f08821872cb4db389110a1efac1d6a7/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:44.119364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d555c-1db1-42d4-ad24-bba073ea6098 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:44.196102 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1020/2006] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:44 2022] PUT /identity/v3/projects/0d30f6fdf7274315ad3566051adddf03/users/eac5cbf36b68469e82a01f263155770c/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:44.196274 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d555c-1db1-42d4-ad24-bba073ea6098 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:44.196955 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3b6a4-5a47-473d-aad5-bcea22740f8d None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:44.197085 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3b6a4-5a47-473d-aad5-bcea22740f8d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:44.197292 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3b6a4-5a47-473d-aad5-bcea22740f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:44.257171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-d1e3b6a4-5a47-473d-aad5-bcea22740f8d None None] MFA Rules not processed for user `3f08821872cb4db389110a1efac1d6a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:44.259036 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d1e3b6a4-5a47-473d-aad5-bcea22740f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:44.332518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad773c-924b-4eb6-9487-b876aeb9bb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:44.333759 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad773c-924b-4eb6-9487-b876aeb9bb34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:44.334119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad773c-924b-4eb6-9487-b876aeb9bb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:44.334475 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad773c-924b-4eb6-9487-b876aeb9bb34 admin admin] PATH_INFO: `/v3/projects/cad5f4f6de7841368ef0b01da91fa55d/users/fede9805b0fa4948a3af31e676aae695/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:44.366158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad773c-924b-4eb6-9487-b876aeb9bb34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cad5f4f6de7841368ef0b01da91fa55d, user_id=fede9805b0fa4948a3af31e676aae695, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:44.369912 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad773c-924b-4eb6-9487-b876aeb9bb34 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:44.383321 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1021/2007] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:44.385460 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567d7d1-aeb6-4bef-822b-0277f62ade3c None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:44.385912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567d7d1-aeb6-4bef-822b-0277f62ade3c None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:44.390232 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567d7d1-aeb6-4bef-822b-0277f62ade3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:44.423175 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-a567d7d1-aeb6-4bef-822b-0277f62ade3c None None] MFA Rules not processed for user `eac5cbf36b68469e82a01f263155770c`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:44.424653 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a567d7d1-aeb6-4bef-822b-0277f62ade3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:44.444629 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 987/2008] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:44 2022] PUT /identity/v3/projects/cad5f4f6de7841368ef0b01da91fa55d/users/fede9805b0fa4948a3af31e676aae695/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:44.445445 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad773c-924b-4eb6-9487-b876aeb9bb34 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:44.527181 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:44.552291 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad773c-924b-4eb6-9487-b876aeb9bb34 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:44.557471 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1022/2009] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:44.558266 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567d7d1-aeb6-4bef-822b-0277f62ade3c None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:44.568979 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6c9cb-d8ae-4890-82ad-91449206e31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:44.569687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6c9cb-d8ae-4890-82ad-91449206e31d admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:44.569972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6c9cb-d8ae-4890-82ad-91449206e31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:44.570315 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6c9cb-d8ae-4890-82ad-91449206e31d admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:44.571061 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6c9cb-d8ae-4890-82ad-91449206e31d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:44.573118 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6c9cb-d8ae-4890-82ad-91449206e31d admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:44.584861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567d7d1-aeb6-4bef-822b-0277f62ade3c None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:44.591435 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d06e84-23c8-4f2f-b81a-40a200511414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:44.592226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d06e84-23c8-4f2f-b81a-40a200511414 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:44.592625 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d06e84-23c8-4f2f-b81a-40a200511414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:44.592944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d06e84-23c8-4f2f-b81a-40a200511414 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:44.593694 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d06e84-23c8-4f2f-b81a-40a200511414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:44.595390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d06e84-23c8-4f2f-b81a-40a200511414 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:44.608506 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1023/2010] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:44.609503 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d06e84-23c8-4f2f-b81a-40a200511414 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:44.621122 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d06e84-23c8-4f2f-b81a-40a200511414 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:44.625959 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b65b3-99a1-4061-9e64-13ef687678e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:44.626642 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b65b3-99a1-4061-9e64-13ef687678e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:44.626990 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b65b3-99a1-4061-9e64-13ef687678e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:44.627292 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b65b3-99a1-4061-9e64-13ef687678e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:44.628010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b65b3-99a1-4061-9e64-13ef687678e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:44.629990 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b65b3-99a1-4061-9e64-13ef687678e3 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:44.672939 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 988/2011] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:44 2022] POST /identity/v3/projects => generated 385 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:44.673901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6c9cb-d8ae-4890-82ad-91449206e31d admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:44.729086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1024/2012] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:44 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:44.734274 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b65b3-99a1-4061-9e64-13ef687678e3 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:44.756310 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6c9cb-d8ae-4890-82ad-91449206e31d admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:44.801714 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b65b3-99a1-4061-9e64-13ef687678e3 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:44.807576 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3e629-7300-44ba-bafe-45cfed862e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:44.808325 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3e629-7300-44ba-bafe-45cfed862e46 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:44.811469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3e629-7300-44ba-bafe-45cfed862e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:44.811469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3e629-7300-44ba-bafe-45cfed862e46 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:44.811469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3e629-7300-44ba-bafe-45cfed862e46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:44.811858 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3e629-7300-44ba-bafe-45cfed862e46 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:44.852288 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37add47c-f0dd-4e7f-adb2-e238df278c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:44.853676 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37add47c-f0dd-4e7f-adb2-e238df278c65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:44.853676 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37add47c-f0dd-4e7f-adb2-e238df278c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:44.854114 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37add47c-f0dd-4e7f-adb2-e238df278c65 admin admin] PATH_INFO: `/v3/projects/cad5f4f6de7841368ef0b01da91fa55d/users/fede9805b0fa4948a3af31e676aae695/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:44.858220 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37add47c-f0dd-4e7f-adb2-e238df278c65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cad5f4f6de7841368ef0b01da91fa55d, user_id=fede9805b0fa4948a3af31e676aae695, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:44.861965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37add47c-f0dd-4e7f-adb2-e238df278c65 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:44.877333 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 989/2013] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:44 2022] PUT /identity/v3/projects/cad5f4f6de7841368ef0b01da91fa55d/users/fede9805b0fa4948a3af31e676aae695/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:44.877938 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37add47c-f0dd-4e7f-adb2-e238df278c65 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:44.892879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1025/2014] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:44 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:44.894260 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ecd97-423f-4958-a3bc-a30c55b0eb5d None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:44.897630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ecd97-423f-4958-a3bc-a30c55b0eb5d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:44.897630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ecd97-423f-4958-a3bc-a30c55b0eb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:44.957246 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-3c7ecd97-423f-4958-a3bc-a30c55b0eb5d None None] MFA Rules not processed for user `fede9805b0fa4948a3af31e676aae695`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:44.962163 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3c7ecd97-423f-4958-a3bc-a30c55b0eb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:44.986273 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37add47c-f0dd-4e7f-adb2-e238df278c65 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:44.991774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b5baf-0f52-4e02-bd76-a22f88922d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:44.994067 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b5baf-0f52-4e02-bd76-a22f88922d2f admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:44.994235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b5baf-0f52-4e02-bd76-a22f88922d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:44.994489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b5baf-0f52-4e02-bd76-a22f88922d2f admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:44.995455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b5baf-0f52-4e02-bd76-a22f88922d2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:44.997588 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b5baf-0f52-4e02-bd76-a22f88922d2f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:45.066077 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1026/2015] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:44 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:45.067635 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ecd97-423f-4958-a3bc-a30c55b0eb5d None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:45.082388 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ecd97-423f-4958-a3bc-a30c55b0eb5d None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:45.091151 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:45.097146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2703915-868a-411b-ba67-e0db7a024774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:45.098575 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2703915-868a-411b-ba67-e0db7a024774 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:45.098869 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2703915-868a-411b-ba67-e0db7a024774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:45.099156 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2703915-868a-411b-ba67-e0db7a024774 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:45.099913 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2703915-868a-411b-ba67-e0db7a024774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:45.100243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 990/2016] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:44 2022] POST /identity/v3/users => generated 297 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:45.101338 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b5baf-0f52-4e02-bd76-a22f88922d2f admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:45.104723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-598b5baf-0f52-4e02-bd76-a22f88922d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:45.107047 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2703915-868a-411b-ba67-e0db7a024774 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:45.120740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1027/2017] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:45.122565 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2703915-868a-411b-ba67-e0db7a024774 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:45.138230 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2703915-868a-411b-ba67-e0db7a024774 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:45.150829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ca43a-0a11-4981-a5cd-319d881217f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:45.151571 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ca43a-0a11-4981-a5cd-319d881217f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:45.151865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ca43a-0a11-4981-a5cd-319d881217f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:45.152148 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ca43a-0a11-4981-a5cd-319d881217f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:45.152923 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ca43a-0a11-4981-a5cd-319d881217f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:45.154592 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ca43a-0a11-4981-a5cd-319d881217f8 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:45.183813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1028/2018] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:45.185783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ca43a-0a11-4981-a5cd-319d881217f8 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:45.220715 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b5baf-0f52-4e02-bd76-a22f88922d2f admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:45.224468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d68fc-266c-415b-a4ac-b809897b0854 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:45.225172 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d68fc-266c-415b-a4ac-b809897b0854 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:45.225458 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d68fc-266c-415b-a4ac-b809897b0854 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:45.225568 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ca43a-0a11-4981-a5cd-319d881217f8 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:45.225740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d68fc-266c-415b-a4ac-b809897b0854 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:45.238701 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d68fc-266c-415b-a4ac-b809897b0854 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:45.240437 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d68fc-266c-415b-a4ac-b809897b0854 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:45.244807 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482610f7-bc5a-4989-98a2-9f88c068014f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:45.254710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482610f7-bc5a-4989-98a2-9f88c068014f admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:45.254710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482610f7-bc5a-4989-98a2-9f88c068014f admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:45.254710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482610f7-bc5a-4989-98a2-9f88c068014f admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:45.254710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482610f7-bc5a-4989-98a2-9f88c068014f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:45.255166 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 991/2019] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:45 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:45.255723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d68fc-266c-415b-a4ac-b809897b0854 service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:45.268609 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482610f7-bc5a-4989-98a2-9f88c068014f admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:45.277244 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d68fc-266c-415b-a4ac-b809897b0854 service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:45.284477 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f7f2a-7e61-40bc-86d2-542df20a70e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:45.285552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f7f2a-7e61-40bc-86d2-542df20a70e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:45.286808 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f7f2a-7e61-40bc-86d2-542df20a70e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:45.286808 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f7f2a-7e61-40bc-86d2-542df20a70e5 admin admin] PATH_INFO: `/v3/projects/1856183a4cf54a829c92d643b1c95d6d/users/e2cc2e31b91b4892a628f89f3ec04ab0/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:45.297983 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1029/2020] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:45.302210 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482610f7-bc5a-4989-98a2-9f88c068014f admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:45.308568 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f7f2a-7e61-40bc-86d2-542df20a70e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1856183a4cf54a829c92d643b1c95d6d, user_id=e2cc2e31b91b4892a628f89f3ec04ab0, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:45.311361 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f7f2a-7e61-40bc-86d2-542df20a70e5 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:45.319224 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482610f7-bc5a-4989-98a2-9f88c068014f admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:45.325959 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8749e-0caf-43b4-9596-d912db8a5095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:45.326856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8749e-0caf-43b4-9596-d912db8a5095 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:45.327140 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8749e-0caf-43b4-9596-d912db8a5095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:45.327443 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8749e-0caf-43b4-9596-d912db8a5095 admin admin] PATH_INFO: `/v3/projects/792adfa3e25441f7955e6b916a9f1ab6/users/5305e6dfae5a443eb08f4ac6cb026cde/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:45.348946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8749e-0caf-43b4-9596-d912db8a5095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=792adfa3e25441f7955e6b916a9f1ab6, user_id=5305e6dfae5a443eb08f4ac6cb026cde, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:45.352288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8749e-0caf-43b4-9596-d912db8a5095 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:45.453481 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 992/2021] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:45 2022] PUT /identity/v3/projects/1856183a4cf54a829c92d643b1c95d6d/users/e2cc2e31b91b4892a628f89f3ec04ab0/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:45.454502 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f7f2a-7e61-40bc-86d2-542df20a70e5 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:45.478321 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1030/2022] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:45 2022] PUT /identity/v3/projects/792adfa3e25441f7955e6b916a9f1ab6/users/5305e6dfae5a443eb08f4ac6cb026cde/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:46:45.499293 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6cec7-e662-4f4b-a0d6-123357edbe83 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:45.499833 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6cec7-e662-4f4b-a0d6-123357edbe83 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:45.500113 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6cec7-e662-4f4b-a0d6-123357edbe83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:45.550980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-29d6cec7-e662-4f4b-a0d6-123357edbe83 None None] MFA Rules not processed for user `5305e6dfae5a443eb08f4ac6cb026cde`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:45.552183 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-29d6cec7-e662-4f4b-a0d6-123357edbe83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:45.583159 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f7f2a-7e61-40bc-86d2-542df20a70e5 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:45.638329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:45.670841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1031/2023] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:45.671820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6cec7-e662-4f4b-a0d6-123357edbe83 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:45.682013 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f904da4-c2de-4871-8b96-868ab723ef24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:45.682731 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f904da4-c2de-4871-8b96-868ab723ef24 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:45.683014 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f904da4-c2de-4871-8b96-868ab723ef24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:45.683289 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f904da4-c2de-4871-8b96-868ab723ef24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:45.684041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f904da4-c2de-4871-8b96-868ab723ef24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:45.685712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f904da4-c2de-4871-8b96-868ab723ef24 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:45.711427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 993/2024] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:45.712830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f904da4-c2de-4871-8b96-868ab723ef24 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:45.734011 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f904da4-c2de-4871-8b96-868ab723ef24 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:45.745484 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091dbf1-8b20-4fea-a524-c4335d079d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:45.746592 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091dbf1-8b20-4fea-a524-c4335d079d4e admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:45.746592 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091dbf1-8b20-4fea-a524-c4335d079d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:45.749002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091dbf1-8b20-4fea-a524-c4335d079d4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:45.754251 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091dbf1-8b20-4fea-a524-c4335d079d4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:45.759452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091dbf1-8b20-4fea-a524-c4335d079d4e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:45.767285 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6cec7-e662-4f4b-a0d6-123357edbe83 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:45.772796 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a9cc4-0ed3-41ed-be0d-372184e232e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:45.774021 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a9cc4-0ed3-41ed-be0d-372184e232e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:45.774494 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a9cc4-0ed3-41ed-be0d-372184e232e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:45.774954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a9cc4-0ed3-41ed-be0d-372184e232e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:45.806797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:45.845038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a9cc4-0ed3-41ed-be0d-372184e232e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:45.846716 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a9cc4-0ed3-41ed-be0d-372184e232e7 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:45.866313 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 994/2025] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:45 2022] POST /identity/v3/projects => generated 387 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:45.869617 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1032/2026] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:45 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:45.871731 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091dbf1-8b20-4fea-a524-c4335d079d4e admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:45.873465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a9cc4-0ed3-41ed-be0d-372184e232e7 service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:45.901330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a9cc4-0ed3-41ed-be0d-372184e232e7 service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:45.978690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:46.019439 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091dbf1-8b20-4fea-a524-c4335d079d4e admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:46.031519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ab364-4cf0-43ad-858e-6535ae2f99f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:46.032568 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ab364-4cf0-43ad-858e-6535ae2f99f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:46.032978 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ab364-4cf0-43ad-858e-6535ae2f99f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:46.033801 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ab364-4cf0-43ad-858e-6535ae2f99f5 admin admin] PATH_INFO: `/v3/projects/1856183a4cf54a829c92d643b1c95d6d/users/e2cc2e31b91b4892a628f89f3ec04ab0/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:46.035898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c51d4b-d4c3-4a74-8795-6e368f4642af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:46.036179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ab364-4cf0-43ad-858e-6535ae2f99f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1856183a4cf54a829c92d643b1c95d6d, user_id=e2cc2e31b91b4892a628f89f3ec04ab0, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:46.038306 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c51d4b-d4c3-4a74-8795-6e368f4642af admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:46.039001 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c51d4b-d4c3-4a74-8795-6e368f4642af admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:46.039261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ab364-4cf0-43ad-858e-6535ae2f99f5 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:46.040412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c51d4b-d4c3-4a74-8795-6e368f4642af admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:46.043019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c51d4b-d4c3-4a74-8795-6e368f4642af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:46.049526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c51d4b-d4c3-4a74-8795-6e368f4642af admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:46.055017 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1033/2027] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:45 2022] PUT /identity/v3/projects/1856183a4cf54a829c92d643b1c95d6d/users/e2cc2e31b91b4892a628f89f3ec04ab0/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:46.056042 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:46.067989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d56230-40f1-44d0-9ad1-db152f3a2e33 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:46.067989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d56230-40f1-44d0-9ad1-db152f3a2e33 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:46.067989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d56230-40f1-44d0-9ad1-db152f3a2e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:46.109640 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-d4d56230-40f1-44d0-9ad1-db152f3a2e33 None None] MFA Rules not processed for user `e2cc2e31b91b4892a628f89f3ec04ab0`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:46.110548 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d4d56230-40f1-44d0-9ad1-db152f3a2e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:46.163080 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 995/2028] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:45 2022] POST /identity/v3/users => generated 300 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:46.167912 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c51d4b-d4c3-4a74-8795-6e368f4642af admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:46.191000 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c51d4b-d4c3-4a74-8795-6e368f4642af admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:46.341617 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1034/2029] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:46:46.389474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e7319-6dd7-470a-a967-2eb17c43b480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:46.391779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e7319-6dd7-470a-a967-2eb17c43b480 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:46.392925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e7319-6dd7-470a-a967-2eb17c43b480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:46.393212 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e7319-6dd7-470a-a967-2eb17c43b480 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:46.393954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e7319-6dd7-470a-a967-2eb17c43b480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:46.400380 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e7319-6dd7-470a-a967-2eb17c43b480 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:46.443590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d56230-40f1-44d0-9ad1-db152f3a2e33 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:46.445007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 996/2030] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:46.457282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d56230-40f1-44d0-9ad1-db152f3a2e33 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:46.467262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17dcbcb-edd5-48b9-9f13-c2586fefad73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:46.480285 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dcbcb-edd5-48b9-9f13-c2586fefad73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:46.480569 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dcbcb-edd5-48b9-9f13-c2586fefad73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:46.483685 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dcbcb-edd5-48b9-9f13-c2586fefad73 admin admin] PATH_INFO: `/v3/projects/353d4225356f4a3cb3d9177a92abf154/users/e8a7a076be024c268556730b7b710579/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:46.540371 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17dcbcb-edd5-48b9-9f13-c2586fefad73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=353d4225356f4a3cb3d9177a92abf154, user_id=e8a7a076be024c268556730b7b710579, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:46.547059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17dcbcb-edd5-48b9-9f13-c2586fefad73 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:46.689204 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1035/2031] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:46 2022] PUT /identity/v3/projects/353d4225356f4a3cb3d9177a92abf154/users/e8a7a076be024c268556730b7b710579/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:46.693020 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e7319-6dd7-470a-a967-2eb17c43b480 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:46.713070 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:46.729582 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e7319-6dd7-470a-a967-2eb17c43b480 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:46.830890 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17dcbcb-edd5-48b9-9f13-c2586fefad73 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:46.842268 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:46.882164 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014c363-fded-4474-b5b8-0a7e966db6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:46.883630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014c363-fded-4474-b5b8-0a7e966db6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:46.884096 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014c363-fded-4474-b5b8-0a7e966db6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:46.884650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014c363-fded-4474-b5b8-0a7e966db6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:46.885620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014c363-fded-4474-b5b8-0a7e966db6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:46.887466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014c363-fded-4474-b5b8-0a7e966db6d5 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:46.917597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 997/2032] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:46.921113 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014c363-fded-4474-b5b8-0a7e966db6d5 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:46.949502 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014c363-fded-4474-b5b8-0a7e966db6d5 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:46.953499 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17dcbcb-edd5-48b9-9f13-c2586fefad73 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:46.955551 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8825d63-9a5e-48f8-b0cf-823dbb4a328c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:46.956551 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8825d63-9a5e-48f8-b0cf-823dbb4a328c admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:46.957020 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8825d63-9a5e-48f8-b0cf-823dbb4a328c admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:46.957195 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8825d63-9a5e-48f8-b0cf-823dbb4a328c admin admin] PATH_INFO: `/v3/projects/353d4225356f4a3cb3d9177a92abf154/users/e8a7a076be024c268556730b7b710579/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:46.958065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1945eb2-770c-46ac-865a-efb2f48efe54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:46.958840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1945eb2-770c-46ac-865a-efb2f48efe54 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:46.959339 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1945eb2-770c-46ac-865a-efb2f48efe54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:46.959538 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8825d63-9a5e-48f8-b0cf-823dbb4a328c admin admin] RBAC: Authorizing `identity:create_grant(project_id=353d4225356f4a3cb3d9177a92abf154, user_id=e8a7a076be024c268556730b7b710579, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:46.959665 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1945eb2-770c-46ac-865a-efb2f48efe54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:46.964553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8825d63-9a5e-48f8-b0cf-823dbb4a328c admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:46.980605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 998/2033] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:46 2022] PUT /identity/v3/projects/353d4225356f4a3cb3d9177a92abf154/users/e8a7a076be024c268556730b7b710579/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:46.991538 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255ae93-4ddd-43bb-9d1c-0ad103f7c243 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:46.991922 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255ae93-4ddd-43bb-9d1c-0ad103f7c243 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:46.992274 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255ae93-4ddd-43bb-9d1c-0ad103f7c243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:47.042916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-8255ae93-4ddd-43bb-9d1c-0ad103f7c243 None None] MFA Rules not processed for user `e8a7a076be024c268556730b7b710579`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:47.054041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-8255ae93-4ddd-43bb-9d1c-0ad103f7c243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:47.064365 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1945eb2-770c-46ac-865a-efb2f48efe54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:47.066077 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1945eb2-770c-46ac-865a-efb2f48efe54 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:47.082983 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1036/2034] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:46:47.190176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 999/2035] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:47.208010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1945eb2-770c-46ac-865a-efb2f48efe54 service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:47.209517 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-f1945eb2-770c-46ac-865a-efb2f48efe54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:47.229476 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1945eb2-770c-46ac-865a-efb2f48efe54 service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:47.322168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83eb2c1-7e49-45b4-9bb9-246f462e78fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:47.323361 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83eb2c1-7e49-45b4-9bb9-246f462e78fd service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:47.323891 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83eb2c1-7e49-45b4-9bb9-246f462e78fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:47.324392 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83eb2c1-7e49-45b4-9bb9-246f462e78fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:47.337295 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83eb2c1-7e49-45b4-9bb9-246f462e78fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:47.339213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83eb2c1-7e49-45b4-9bb9-246f462e78fd service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:47.350458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1037/2036] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:47.389477 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255ae93-4ddd-43bb-9d1c-0ad103f7c243 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:47.403940 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255ae93-4ddd-43bb-9d1c-0ad103f7c243 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:47.408390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05611b5d-bce3-4120-8843-0c80732e3b80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:47.409186 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05611b5d-bce3-4120-8843-0c80732e3b80 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:47.409477 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05611b5d-bce3-4120-8843-0c80732e3b80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:47.409924 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05611b5d-bce3-4120-8843-0c80732e3b80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:47.446845 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:47.501687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05611b5d-bce3-4120-8843-0c80732e3b80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:47.503754 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05611b5d-bce3-4120-8843-0c80732e3b80 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:47.529320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1000/2037] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:47.680471 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:48.115554 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83eb2c1-7e49-45b4-9bb9-246f462e78fd service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:48.224928 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83eb2c1-7e49-45b4-9bb9-246f462e78fd service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:48.230486 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b4b58-a77b-4810-9b07-3054f0ec6001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:48.231530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b4b58-a77b-4810-9b07-3054f0ec6001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:48.232021 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b4b58-a77b-4810-9b07-3054f0ec6001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:48.232478 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b4b58-a77b-4810-9b07-3054f0ec6001 admin admin] PATH_INFO: `/v3/users/eac5cbf36b68469e82a01f263155770c` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:48.235133 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b4b58-a77b-4810-9b07-3054f0ec6001 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eac5cbf36b68469e82a01f263155770c)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:48.237767 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b4b58-a77b-4810-9b07-3054f0ec6001 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:48.326093 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:48.556771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-840b4b58-a77b-4810-9b07-3054f0ec6001 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eac5cbf36b68469e82a01f263155770c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:48.572327 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-840b4b58-a77b-4810-9b07-3054f0ec6001 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eac5cbf36b68469e82a01f263155770c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:48.573677 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-840b4b58-a77b-4810-9b07-3054f0ec6001 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eac5cbf36b68469e82a01f263155770c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:48.574363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-840b4b58-a77b-4810-9b07-3054f0ec6001 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eac5cbf36b68469e82a01f263155770c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:48.583440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-840b4b58-a77b-4810-9b07-3054f0ec6001 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eac5cbf36b68469e82a01f263155770c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:48.704047 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1038/2038] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:48 2022] DELETE /identity/v3/users/eac5cbf36b68469e82a01f263155770c => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:48.712101 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d54bca-c7a0-4993-b074-e435a8a747dd None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:48.713867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d54bca-c7a0-4993-b074-e435a8a747dd None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:48.714237 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d54bca-c7a0-4993-b074-e435a8a747dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:48.740775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-b1d54bca-c7a0-4993-b074-e435a8a747dd None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:48.740775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-b1d54bca-c7a0-4993-b074-e435a8a747dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:48.754577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b4b58-a77b-4810-9b07-3054f0ec6001 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:48.756065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-840b4b58-a77b-4810-9b07-3054f0ec6001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:48.805762 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1001/2039] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:48.816334 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d54bca-c7a0-4993-b074-e435a8a747dd None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:48.821562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-b1d54bca-c7a0-4993-b074-e435a8a747dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:48.831479 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b4b58-a77b-4810-9b07-3054f0ec6001 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:48.835267 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80e19b-3a5b-4ce4-8937-ce5dad3f5bf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:48.835918 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80e19b-3a5b-4ce4-8937-ce5dad3f5bf8 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:48.836296 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80e19b-3a5b-4ce4-8937-ce5dad3f5bf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:48.836612 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80e19b-3a5b-4ce4-8937-ce5dad3f5bf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:48.838563 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ac80e19b-3a5b-4ce4-8937-ce5dad3f5bf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:48.897725 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:48.913577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d54bca-c7a0-4993-b074-e435a8a747dd None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:48.918758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446b278-d2b7-49e9-a6d8-518aa0d04623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:48.919531 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446b278-d2b7-49e9-a6d8-518aa0d04623 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:48.919822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446b278-d2b7-49e9-a6d8-518aa0d04623 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:48.920157 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446b278-d2b7-49e9-a6d8-518aa0d04623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:48.943302 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446b278-d2b7-49e9-a6d8-518aa0d04623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:48.945920 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446b278-d2b7-49e9-a6d8-518aa0d04623 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:48.950218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80e19b-3a5b-4ce4-8937-ce5dad3f5bf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:48.951564 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80e19b-3a5b-4ce4-8937-ce5dad3f5bf8 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:48.966382 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1002/2040] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:48.973917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1039/2041] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:46:49.612096 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:49.794445 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446b278-d2b7-49e9-a6d8-518aa0d04623 service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:49.799390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-b446b278-d2b7-49e9-a6d8-518aa0d04623 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:49.838652 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446b278-d2b7-49e9-a6d8-518aa0d04623 service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:49.846416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23640cd-78ef-481e-a379-431f7a8d3412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:49.848652 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23640cd-78ef-481e-a379-431f7a8d3412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:49.848971 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23640cd-78ef-481e-a379-431f7a8d3412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:49.849228 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23640cd-78ef-481e-a379-431f7a8d3412 admin admin] PATH_INFO: `/v3/projects/0d30f6fdf7274315ad3566051adddf03` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:49.852635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23640cd-78ef-481e-a379-431f7a8d3412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d30f6fdf7274315ad3566051adddf03)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:49.855232 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23640cd-78ef-481e-a379-431f7a8d3412 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:49.935354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80e19b-3a5b-4ce4-8937-ce5dad3f5bf8 service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:49.936940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ac80e19b-3a5b-4ce4-8937-ce5dad3f5bf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:49.955983 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80e19b-3a5b-4ce4-8937-ce5dad3f5bf8 service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:49.963634 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-b23640cd-78ef-481e-a379-431f7a8d3412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d30f6fdf7274315ad3566051adddf03', 'request_id': 'req-b23640cd-78ef-481e-a379-431f7a8d3412'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:50.018515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07228d-233e-4187-996f-54c0ffdcd06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:50.019463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07228d-233e-4187-996f-54c0ffdcd06a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:50.019761 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07228d-233e-4187-996f-54c0ffdcd06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:50.020042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07228d-233e-4187-996f-54c0ffdcd06a admin admin] PATH_INFO: `/v3/users/365b889f8f1a4c5dad09364e5df705ba` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:50.021303 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf07228d-233e-4187-996f-54c0ffdcd06a admin admin] RBAC: Authorizing `identity:delete_user(user_id=365b889f8f1a4c5dad09364e5df705ba)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:50.027849 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf07228d-233e-4187-996f-54c0ffdcd06a admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:50.045530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-b23640cd-78ef-481e-a379-431f7a8d3412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d30f6fdf7274315ad3566051adddf03', 'request_id': 'req-b23640cd-78ef-481e-a379-431f7a8d3412'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:50.052722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-b23640cd-78ef-481e-a379-431f7a8d3412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d30f6fdf7274315ad3566051adddf03', 'request_id': 'req-b23640cd-78ef-481e-a379-431f7a8d3412'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:50.064565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-b23640cd-78ef-481e-a379-431f7a8d3412 admin admin] The token cache is being invalidate because project 0d30f6fdf7274315ad3566051adddf03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:50.064878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-b23640cd-78ef-481e-a379-431f7a8d3412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:50.065570 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-b23640cd-78ef-481e-a379-431f7a8d3412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:50.067178 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1003/2042] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:49 2022] DELETE /identity/v3/projects/0d30f6fdf7274315ad3566051adddf03 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:50.071128 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23640cd-78ef-481e-a379-431f7a8d3412 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:50.072773 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-b23640cd-78ef-481e-a379-431f7a8d3412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:50.085552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23640cd-78ef-481e-a379-431f7a8d3412 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:50.091128 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc2232-9f94-448f-9bb5-210c68dea3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:50.092001 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc2232-9f94-448f-9bb5-210c68dea3f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:50.097158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc2232-9f94-448f-9bb5-210c68dea3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:50.097158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc2232-9f94-448f-9bb5-210c68dea3f2 admin admin] PATH_INFO: `/v3/users/e2cc2e31b91b4892a628f89f3ec04ab0` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:50.097158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc2232-9f94-448f-9bb5-210c68dea3f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2cc2e31b91b4892a628f89f3ec04ab0)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:50.097158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc2232-9f94-448f-9bb5-210c68dea3f2 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:50.232469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-bf07228d-233e-4187-996f-54c0ffdcd06a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '365b889f8f1a4c5dad09364e5df705ba'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:50.239928 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-bf07228d-233e-4187-996f-54c0ffdcd06a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '365b889f8f1a4c5dad09364e5df705ba'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:50.240722 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-bf07228d-233e-4187-996f-54c0ffdcd06a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '365b889f8f1a4c5dad09364e5df705ba'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:50.241342 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-bf07228d-233e-4187-996f-54c0ffdcd06a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '365b889f8f1a4c5dad09364e5df705ba'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:50.248871 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-bf07228d-233e-4187-996f-54c0ffdcd06a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '365b889f8f1a4c5dad09364e5df705ba'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:50.354415 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1040/2043] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:49 2022] DELETE /identity/v3/users/365b889f8f1a4c5dad09364e5df705ba => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:50.393184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a1dc2232-9f94-448f-9bb5-210c68dea3f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2cc2e31b91b4892a628f89f3ec04ab0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:50.407811 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a1dc2232-9f94-448f-9bb5-210c68dea3f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2cc2e31b91b4892a628f89f3ec04ab0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:50.480560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a1dc2232-9f94-448f-9bb5-210c68dea3f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2cc2e31b91b4892a628f89f3ec04ab0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:50.481246 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a1dc2232-9f94-448f-9bb5-210c68dea3f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2cc2e31b91b4892a628f89f3ec04ab0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:50.483068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a1dc2232-9f94-448f-9bb5-210c68dea3f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2cc2e31b91b4892a628f89f3ec04ab0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:50.493653 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:50.508697 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1004/2044] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:50 2022] DELETE /identity/v3/users/e2cc2e31b91b4892a628f89f3ec04ab0 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:46:50.643107 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07228d-233e-4187-996f-54c0ffdcd06a admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:50.644607 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-bf07228d-233e-4187-996f-54c0ffdcd06a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:50.722942 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07228d-233e-4187-996f-54c0ffdcd06a admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:50.780963 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c522407-a919-453f-9995-1fa265d6d1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:50.781954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c522407-a919-453f-9995-1fa265d6d1a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:50.782238 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c522407-a919-453f-9995-1fa265d6d1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:50.782516 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c522407-a919-453f-9995-1fa265d6d1a9 admin admin] PATH_INFO: `/v3/projects/a35daf906b2e442c9b27cb6f83673f0a` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:50.783834 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c522407-a919-453f-9995-1fa265d6d1a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a35daf906b2e442c9b27cb6f83673f0a)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:50.785835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c522407-a919-453f-9995-1fa265d6d1a9 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:50.934761 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6c522407-a919-453f-9995-1fa265d6d1a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a35daf906b2e442c9b27cb6f83673f0a', 'request_id': 'req-6c522407-a919-453f-9995-1fa265d6d1a9'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:50.941094 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6c522407-a919-453f-9995-1fa265d6d1a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a35daf906b2e442c9b27cb6f83673f0a', 'request_id': 'req-6c522407-a919-453f-9995-1fa265d6d1a9'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:50.947792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6c522407-a919-453f-9995-1fa265d6d1a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a35daf906b2e442c9b27cb6f83673f0a', 'request_id': 'req-6c522407-a919-453f-9995-1fa265d6d1a9'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:50.996440 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc2232-9f94-448f-9bb5-210c68dea3f2 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:50.999408 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-a1dc2232-9f94-448f-9bb5-210c68dea3f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:51.024364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc2232-9f94-448f-9bb5-210c68dea3f2 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:51.034728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6c522407-a919-453f-9995-1fa265d6d1a9 admin admin] The token cache is being invalidate because project a35daf906b2e442c9b27cb6f83673f0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:51.035210 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6c522407-a919-453f-9995-1fa265d6d1a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:51.036054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6c522407-a919-453f-9995-1fa265d6d1a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:51.037971 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1041/2045] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:50 2022] DELETE /identity/v3/projects/a35daf906b2e442c9b27cb6f83673f0a => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:51.043789 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b5e9a-0ad1-41a3-be1e-911204504feb None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:51.044177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b5e9a-0ad1-41a3-be1e-911204504feb None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:51.044536 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b5e9a-0ad1-41a3-be1e-911204504feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:51.065465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-ff9b5e9a-0ad1-41a3-be1e-911204504feb None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:51.066702 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ff9b5e9a-0ad1-41a3-be1e-911204504feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:51.088925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa48648-83cd-4e55-b212-3fa6da415f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:51.089832 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa48648-83cd-4e55-b212-3fa6da415f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:51.090117 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa48648-83cd-4e55-b212-3fa6da415f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:51.090400 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa48648-83cd-4e55-b212-3fa6da415f6b admin admin] PATH_INFO: `/v3/projects/1856183a4cf54a829c92d643b1c95d6d` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:51.091594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa48648-83cd-4e55-b212-3fa6da415f6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1856183a4cf54a829c92d643b1c95d6d)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:51.093540 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa48648-83cd-4e55-b212-3fa6da415f6b admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:51.115445 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:51.136055 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1042/2046] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:51.144057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f360a-754c-4d20-93a1-081649110546 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:51.144354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f360a-754c-4d20-93a1-081649110546 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:51.144677 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f360a-754c-4d20-93a1-081649110546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:51.163135 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-8c8f360a-754c-4d20-93a1-081649110546 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:51.164605 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-8c8f360a-754c-4d20-93a1-081649110546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:51.172460 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1043/2047] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:51.177122 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f360a-754c-4d20-93a1-081649110546 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:51.186582 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f360a-754c-4d20-93a1-081649110546 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:51.191304 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bb03c-867e-45dd-a000-87850a5ca796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:51.191955 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bb03c-867e-45dd-a000-87850a5ca796 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:51.192244 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bb03c-867e-45dd-a000-87850a5ca796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:51.192522 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bb03c-867e-45dd-a000-87850a5ca796 admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:51.193284 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bb03c-867e-45dd-a000-87850a5ca796 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:46:51.193593 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bb03c-867e-45dd-a000-87850a5ca796 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:51.195208 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bb03c-867e-45dd-a000-87850a5ca796 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:51.208274 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1044/2048] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:46:51.213485 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bb03c-867e-45dd-a000-87850a5ca796 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:51.223668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bb03c-867e-45dd-a000-87850a5ca796 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:51.229106 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ea286-27d9-4605-a2c5-f6687ea78f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:51.230818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ea286-27d9-4605-a2c5-f6687ea78f42 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:51.230818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ea286-27d9-4605-a2c5-f6687ea78f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:51.230818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ea286-27d9-4605-a2c5-f6687ea78f42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:51.231090 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ea286-27d9-4605-a2c5-f6687ea78f42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:51.233298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ea286-27d9-4605-a2c5-f6687ea78f42 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:51.255534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-5aa48648-83cd-4e55-b212-3fa6da415f6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1856183a4cf54a829c92d643b1c95d6d', 'request_id': 'req-5aa48648-83cd-4e55-b212-3fa6da415f6b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:51.328140 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1045/2049] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:51 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:46:51.332180 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ea286-27d9-4605-a2c5-f6687ea78f42 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:51.342680 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-5aa48648-83cd-4e55-b212-3fa6da415f6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1856183a4cf54a829c92d643b1c95d6d', 'request_id': 'req-5aa48648-83cd-4e55-b212-3fa6da415f6b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:51.349688 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-5aa48648-83cd-4e55-b212-3fa6da415f6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1856183a4cf54a829c92d643b1c95d6d', 'request_id': 'req-5aa48648-83cd-4e55-b212-3fa6da415f6b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:51.361987 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-5aa48648-83cd-4e55-b212-3fa6da415f6b admin admin] The token cache is being invalidate because project 1856183a4cf54a829c92d643b1c95d6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:51.362362 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-5aa48648-83cd-4e55-b212-3fa6da415f6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:51.363119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-5aa48648-83cd-4e55-b212-3fa6da415f6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:51.364953 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1005/2050] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:50 2022] DELETE /identity/v3/projects/1856183a4cf54a829c92d643b1c95d6d => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:51.372203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe2dc6-0d15-40d0-95a8-078cf91922c1 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:51.372732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe2dc6-0d15-40d0-95a8-078cf91922c1 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:51.373145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe2dc6-0d15-40d0-95a8-078cf91922c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:51.404460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-ebbe2dc6-0d15-40d0-95a8-078cf91922c1 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:51.404460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ebbe2dc6-0d15-40d0-95a8-078cf91922c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:51.412866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ea286-27d9-4605-a2c5-f6687ea78f42 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:51.417857 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74508d-de65-46b5-afb5-552ecf1e29b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:51.418537 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74508d-de65-46b5-afb5-552ecf1e29b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:51.420689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74508d-de65-46b5-afb5-552ecf1e29b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:51.420689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74508d-de65-46b5-afb5-552ecf1e29b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:51.420689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74508d-de65-46b5-afb5-552ecf1e29b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:51.421726 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74508d-de65-46b5-afb5-552ecf1e29b6 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:51.461744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1006/2051] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:51.471548 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ec8f3-12b2-4646-ba3c-7ee49ad41fcc None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:51.471840 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ec8f3-12b2-4646-ba3c-7ee49ad41fcc None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:51.472124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ec8f3-12b2-4646-ba3c-7ee49ad41fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:51.489982 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-c79ec8f3-12b2-4646-ba3c-7ee49ad41fcc None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:51.491119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c79ec8f3-12b2-4646-ba3c-7ee49ad41fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:51.500000 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1007/2052] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:46:51.505701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1046/2053] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:51 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:46:51.506760 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ec8f3-12b2-4646-ba3c-7ee49ad41fcc None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:51.506947 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74508d-de65-46b5-afb5-552ecf1e29b6 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:51.508780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c79ec8f3-12b2-4646-ba3c-7ee49ad41fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:51.518154 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74508d-de65-46b5-afb5-552ecf1e29b6 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:51.521029 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ec8f3-12b2-4646-ba3c-7ee49ad41fcc None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:51.523269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fdb7b3-3bc6-4114-8eca-90aa3fad81a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:51.524091 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdb7b3-3bc6-4114-8eca-90aa3fad81a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:51.524374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdb7b3-3bc6-4114-8eca-90aa3fad81a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:51.524680 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdb7b3-3bc6-4114-8eca-90aa3fad81a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:51.529884 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fdb7b3-3bc6-4114-8eca-90aa3fad81a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:46:51.529884 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fdb7b3-3bc6-4114-8eca-90aa3fad81a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:51.530127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b1a22-93f9-4238-9b6a-188aa8b6944f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:51.530127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b1a22-93f9-4238-9b6a-188aa8b6944f admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:51.530308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fdb7b3-3bc6-4114-8eca-90aa3fad81a1 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:51.530407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b1a22-93f9-4238-9b6a-188aa8b6944f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:51.530407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b1a22-93f9-4238-9b6a-188aa8b6944f admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:51.530407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b1a22-93f9-4238-9b6a-188aa8b6944f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:51.530407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b1a22-93f9-4238-9b6a-188aa8b6944f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:51.540914 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1008/2054] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:46:51.546832 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1047/2055] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:46:51.546962 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b1a22-93f9-4238-9b6a-188aa8b6944f admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:51.547786 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fdb7b3-3bc6-4114-8eca-90aa3fad81a1 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:51.555384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b1a22-93f9-4238-9b6a-188aa8b6944f admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:51.558739 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fdb7b3-3bc6-4114-8eca-90aa3fad81a1 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:51.562524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b367be7-4856-4b04-9faf-3310dfed98c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:51.562524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b367be7-4856-4b04-9faf-3310dfed98c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:51.562524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b367be7-4856-4b04-9faf-3310dfed98c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:51.562524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b367be7-4856-4b04-9faf-3310dfed98c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:51.563096 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b367be7-4856-4b04-9faf-3310dfed98c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:51.563802 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b367be7-4856-4b04-9faf-3310dfed98c8 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:51.572847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ac045-a8c7-48f8-8b8c-145068f04b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:51.573496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ac045-a8c7-48f8-8b8c-145068f04b25 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:51.573987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ac045-a8c7-48f8-8b8c-145068f04b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:51.574407 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ac045-a8c7-48f8-8b8c-145068f04b25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:51.575182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ac045-a8c7-48f8-8b8c-145068f04b25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:51.576683 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1009/2056] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:51.577459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ac045-a8c7-48f8-8b8c-145068f04b25 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:51.580773 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b367be7-4856-4b04-9faf-3310dfed98c8 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:51.595271 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b367be7-4856-4b04-9faf-3310dfed98c8 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:51.600513 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e7571-bdb5-4b02-a9b1-15a48aadeda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:51.601674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e7571-bdb5-4b02-a9b1-15a48aadeda7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:51.606516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e7571-bdb5-4b02-a9b1-15a48aadeda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:51.606516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e7571-bdb5-4b02-a9b1-15a48aadeda7 admin admin] PATH_INFO: `/v3/projects/c0492357de44424887f2613db6868b8a/users/9d375c5d5e2e400c8b95ffdf39433faf/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:51.620809 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e7571-bdb5-4b02-a9b1-15a48aadeda7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0492357de44424887f2613db6868b8a, user_id=9d375c5d5e2e400c8b95ffdf39433faf, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:51.623548 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e7571-bdb5-4b02-a9b1-15a48aadeda7 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:51.663905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1048/2057] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:51 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:46:51.668026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ac045-a8c7-48f8-8b8c-145068f04b25 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:51.681119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1010/2058] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:51 2022] PUT /identity/v3/projects/c0492357de44424887f2613db6868b8a/users/9d375c5d5e2e400c8b95ffdf39433faf/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:46:51.696325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218fab3-24c5-4e43-a5fd-4fbf29c7f2af None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:51.696325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218fab3-24c5-4e43-a5fd-4fbf29c7f2af None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:51.696689 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218fab3-24c5-4e43-a5fd-4fbf29c7f2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:51.729309 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-4218fab3-24c5-4e43-a5fd-4fbf29c7f2af None None] MFA Rules not processed for user `9d375c5d5e2e400c8b95ffdf39433faf`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:51.730368 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-4218fab3-24c5-4e43-a5fd-4fbf29c7f2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:51.733535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ac045-a8c7-48f8-8b8c-145068f04b25 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:51.804409 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8e472-7335-4f2c-8a88-d3948ab1f310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:51.808958 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8e472-7335-4f2c-8a88-d3948ab1f310 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:51.808958 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8e472-7335-4f2c-8a88-d3948ab1f310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:51.808958 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8e472-7335-4f2c-8a88-d3948ab1f310 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:51.808958 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8e472-7335-4f2c-8a88-d3948ab1f310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:51.809351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8e472-7335-4f2c-8a88-d3948ab1f310 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:51.828117 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1011/2059] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:46:51.832819 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218fab3-24c5-4e43-a5fd-4fbf29c7f2af None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:51.847320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218fab3-24c5-4e43-a5fd-4fbf29c7f2af None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:51.852594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c7cc0-941e-4028-8552-7f7e75ef946f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:51.853652 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c7cc0-941e-4028-8552-7f7e75ef946f admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:51.853942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c7cc0-941e-4028-8552-7f7e75ef946f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:51.854225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c7cc0-941e-4028-8552-7f7e75ef946f admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:51.855115 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c7cc0-941e-4028-8552-7f7e75ef946f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:51.857080 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c7cc0-941e-4028-8552-7f7e75ef946f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:51.904939 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1049/2060] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:51 2022] POST /identity/v3/users => generated 290 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:46:51.909080 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8e472-7335-4f2c-8a88-d3948ab1f310 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:51.919809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8e472-7335-4f2c-8a88-d3948ab1f310 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:51.924433 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9c3e2-4f01-4edb-b414-ad881860ff86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:51.925158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9c3e2-4f01-4edb-b414-ad881860ff86 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:51.925447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9c3e2-4f01-4edb-b414-ad881860ff86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:51.925757 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9c3e2-4f01-4edb-b414-ad881860ff86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:51.926496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9c3e2-4f01-4edb-b414-ad881860ff86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:51.928131 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9c3e2-4f01-4edb-b414-ad881860ff86 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:51.939239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1050/2061] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:51.940110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9c3e2-4f01-4edb-b414-ad881860ff86 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:51.941824 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-59c9c3e2-4f01-4edb-b414-ad881860ff86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:51.955489 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9c3e2-4f01-4edb-b414-ad881860ff86 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:51.960484 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c638853-16ab-4aab-ae54-b8939f486d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:51.962031 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c638853-16ab-4aab-ae54-b8939f486d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:51.962312 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c638853-16ab-4aab-ae54-b8939f486d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:51.962589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c638853-16ab-4aab-ae54-b8939f486d00 admin admin] PATH_INFO: `/v3/users/42e087686e8d4af48140c53bb583add3` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:51.963793 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c638853-16ab-4aab-ae54-b8939f486d00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42e087686e8d4af48140c53bb583add3)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:51.965741 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c638853-16ab-4aab-ae54-b8939f486d00 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:51.987280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1012/2062] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:51 2022] POST /identity/v3/projects => generated 371 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:51.987280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c7cc0-941e-4028-8552-7f7e75ef946f admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:52.049349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c7cc0-941e-4028-8552-7f7e75ef946f admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:52.053934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f10df7-d11b-40e7-81fd-f49660da2411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:52.054652 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f10df7-d11b-40e7-81fd-f49660da2411 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:52.054940 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f10df7-d11b-40e7-81fd-f49660da2411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:52.055215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f10df7-d11b-40e7-81fd-f49660da2411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:52.055942 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f10df7-d11b-40e7-81fd-f49660da2411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:52.057572 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f10df7-d11b-40e7-81fd-f49660da2411 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:52.068040 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1013/2063] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:52.068993 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f10df7-d11b-40e7-81fd-f49660da2411 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:52.079576 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f10df7-d11b-40e7-81fd-f49660da2411 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:52.084139 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897f924-2115-4896-b057-2591830fc0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:52.084823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897f924-2115-4896-b057-2591830fc0d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:52.085125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897f924-2115-4896-b057-2591830fc0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:52.085402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897f924-2115-4896-b057-2591830fc0d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:52.086715 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897f924-2115-4896-b057-2591830fc0d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:52.088636 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897f924-2115-4896-b057-2591830fc0d4 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:52.162457 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2c638853-16ab-4aab-ae54-b8939f486d00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42e087686e8d4af48140c53bb583add3'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:52.175214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2c638853-16ab-4aab-ae54-b8939f486d00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42e087686e8d4af48140c53bb583add3'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:52.175944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2c638853-16ab-4aab-ae54-b8939f486d00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42e087686e8d4af48140c53bb583add3'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:52.176599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2c638853-16ab-4aab-ae54-b8939f486d00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42e087686e8d4af48140c53bb583add3'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:52.177523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1014/2064] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:52 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:52.178415 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897f924-2115-4896-b057-2591830fc0d4 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:52.180736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-9897f924-2115-4896-b057-2591830fc0d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:52.187940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2c638853-16ab-4aab-ae54-b8939f486d00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42e087686e8d4af48140c53bb583add3'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:52.193071 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897f924-2115-4896-b057-2591830fc0d4 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:52.200599 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3efd4fe-b760-44f7-9455-38bea5875f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:52.201520 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efd4fe-b760-44f7-9455-38bea5875f87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:52.201856 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efd4fe-b760-44f7-9455-38bea5875f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:52.202123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efd4fe-b760-44f7-9455-38bea5875f87 admin admin] PATH_INFO: `/v3/projects/cb1c26e7fcbd4c0c9bb4a68e54569e3a/users/dd98b2a31e4c46599269684782915703/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:52.219545 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3efd4fe-b760-44f7-9455-38bea5875f87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb1c26e7fcbd4c0c9bb4a68e54569e3a, user_id=dd98b2a31e4c46599269684782915703, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:52.226219 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3efd4fe-b760-44f7-9455-38bea5875f87 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:52.292787 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1051/2065] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:51 2022] DELETE /identity/v3/users/42e087686e8d4af48140c53bb583add3 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:52.297700 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c638853-16ab-4aab-ae54-b8939f486d00 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:52.299169 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-2c638853-16ab-4aab-ae54-b8939f486d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:52.321629 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1015/2066] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:52 2022] PUT /identity/v3/projects/cb1c26e7fcbd4c0c9bb4a68e54569e3a/users/dd98b2a31e4c46599269684782915703/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:52.321629 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a80dd9-5ba4-4929-8814-57323ee3af97 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:52.321629 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a80dd9-5ba4-4929-8814-57323ee3af97 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:52.321629 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a80dd9-5ba4-4929-8814-57323ee3af97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:52.348151 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-78a80dd9-5ba4-4929-8814-57323ee3af97 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:52.348151 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-78a80dd9-5ba4-4929-8814-57323ee3af97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:52.365584 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:52.392727 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c638853-16ab-4aab-ae54-b8939f486d00 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:52.454096 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1016/2067] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:52.455444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ff778-cb6c-4575-9239-768aa14b7827 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:52.455741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ff778-cb6c-4575-9239-768aa14b7827 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:52.456027 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ff778-cb6c-4575-9239-768aa14b7827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:52.486947 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-e18ff778-cb6c-4575-9239-768aa14b7827 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:52.488166 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e18ff778-cb6c-4575-9239-768aa14b7827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:52.490621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5741a4-4ac7-450f-bcdc-3c50f063872e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:52.493851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5741a4-4ac7-450f-bcdc-3c50f063872e admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:52.493851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5741a4-4ac7-450f-bcdc-3c50f063872e admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:52.494075 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5741a4-4ac7-450f-bcdc-3c50f063872e admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:52.495343 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5741a4-4ac7-450f-bcdc-3c50f063872e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:52.496753 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5741a4-4ac7-450f-bcdc-3c50f063872e admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:52.514270 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1052/2068] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:52.514270 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5741a4-4ac7-450f-bcdc-3c50f063872e admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:52.514635 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:52.516410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-bf5741a4-4ac7-450f-bcdc-3c50f063872e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:52.519592 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1017/2069] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:52.519592 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ff778-cb6c-4575-9239-768aa14b7827 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:52.528992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ff778-cb6c-4575-9239-768aa14b7827 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:52.537784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d8022-39b9-4684-895e-8eb428ae7d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:52.540616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d8022-39b9-4684-895e-8eb428ae7d83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:52.540616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d8022-39b9-4684-895e-8eb428ae7d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:52.540616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d8022-39b9-4684-895e-8eb428ae7d83 admin admin] PATH_INFO: `/v3/projects/ed48a88e673e4716ae7a65f5b8318a2b/users/42d52b44c7aa4d1c843780e9d47dc5f6/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:52.561637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d8022-39b9-4684-895e-8eb428ae7d83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed48a88e673e4716ae7a65f5b8318a2b, user_id=42d52b44c7aa4d1c843780e9d47dc5f6, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:52.564562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d8022-39b9-4684-895e-8eb428ae7d83 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:52.587145 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5741a4-4ac7-450f-bcdc-3c50f063872e admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:52.592871 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54270304-6c5a-4b10-b6c5-b7eeaee8d0be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:52.593530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54270304-6c5a-4b10-b6c5-b7eeaee8d0be service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:52.593890 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54270304-6c5a-4b10-b6c5-b7eeaee8d0be service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:52.594169 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54270304-6c5a-4b10-b6c5-b7eeaee8d0be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:52.604412 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54270304-6c5a-4b10-b6c5-b7eeaee8d0be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:52.605891 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54270304-6c5a-4b10-b6c5-b7eeaee8d0be service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:52.615709 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1053/2070] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:52.616544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54270304-6c5a-4b10-b6c5-b7eeaee8d0be service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:52.618717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-54270304-6c5a-4b10-b6c5-b7eeaee8d0be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:52.630316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54270304-6c5a-4b10-b6c5-b7eeaee8d0be service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:52.635075 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99073a9d-1105-4561-89af-7ec752dedc19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:52.635958 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99073a9d-1105-4561-89af-7ec752dedc19 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:52.636242 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99073a9d-1105-4561-89af-7ec752dedc19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:52.637218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99073a9d-1105-4561-89af-7ec752dedc19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:52.646648 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99073a9d-1105-4561-89af-7ec752dedc19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:52.648107 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99073a9d-1105-4561-89af-7ec752dedc19 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:52.663447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1018/2071] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:52 2022] PUT /identity/v3/projects/ed48a88e673e4716ae7a65f5b8318a2b/users/42d52b44c7aa4d1c843780e9d47dc5f6/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:52.667728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1054/2072] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:52.668191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d8022-39b9-4684-895e-8eb428ae7d83 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:52.735971 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d8022-39b9-4684-895e-8eb428ae7d83 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:52.740654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd70c8-7a9e-44f8-9d09-969398c0fe76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:52.776688 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd70c8-7a9e-44f8-9d09-969398c0fe76 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:52.776688 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd70c8-7a9e-44f8-9d09-969398c0fe76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:52.776688 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd70c8-7a9e-44f8-9d09-969398c0fe76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:52.776688 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd70c8-7a9e-44f8-9d09-969398c0fe76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:52.776688 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd70c8-7a9e-44f8-9d09-969398c0fe76 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:52.776688 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1019/2073] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:52.781071 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99073a9d-1105-4561-89af-7ec752dedc19 service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:52.792677 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99073a9d-1105-4561-89af-7ec752dedc19 service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:52.797432 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd3e57-a16a-4cb4-856d-c83623548ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:52.798371 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd3e57-a16a-4cb4-856d-c83623548ade admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:52.798658 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd3e57-a16a-4cb4-856d-c83623548ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:52.798939 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd3e57-a16a-4cb4-856d-c83623548ade admin admin] PATH_INFO: `/v3/projects/ed48a88e673e4716ae7a65f5b8318a2b/users/42d52b44c7aa4d1c843780e9d47dc5f6/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:52.800975 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd3e57-a16a-4cb4-856d-c83623548ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed48a88e673e4716ae7a65f5b8318a2b, user_id=42d52b44c7aa4d1c843780e9d47dc5f6, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:52.803696 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd3e57-a16a-4cb4-856d-c83623548ade admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:52.818251 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1055/2074] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:52 2022] PUT /identity/v3/projects/ed48a88e673e4716ae7a65f5b8318a2b/users/42d52b44c7aa4d1c843780e9d47dc5f6/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:46:52.828914 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4a940-46f9-402b-b4a9-693c76e66d94 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:52.829215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4a940-46f9-402b-b4a9-693c76e66d94 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:52.829502 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4a940-46f9-402b-b4a9-693c76e66d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:52.859023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-94b4a940-46f9-402b-b4a9-693c76e66d94 None None] MFA Rules not processed for user `42d52b44c7aa4d1c843780e9d47dc5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:52.860169 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-94b4a940-46f9-402b-b4a9-693c76e66d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:52.933375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1020/2075] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:52 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:52.946365 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd3e57-a16a-4cb4-856d-c83623548ade admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:52.948069 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-87bd3e57-a16a-4cb4-856d-c83623548ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:53.013506 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd3e57-a16a-4cb4-856d-c83623548ade admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:53.018858 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96d319-a815-4259-b60c-dfbf4811d7c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:53.019555 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96d319-a815-4259-b60c-dfbf4811d7c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:53.019858 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96d319-a815-4259-b60c-dfbf4811d7c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:53.020115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96d319-a815-4259-b60c-dfbf4811d7c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:53.031687 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96d319-a815-4259-b60c-dfbf4811d7c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:53.033226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96d319-a815-4259-b60c-dfbf4811d7c1 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:53.046763 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1056/2076] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:52 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:53.356391 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4a940-46f9-402b-b4a9-693c76e66d94 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:53.356391 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-94b4a940-46f9-402b-b4a9-693c76e66d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:53.432767 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4a940-46f9-402b-b4a9-693c76e66d94 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:53.437685 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e9c30-4941-4cbc-8398-08c6c407c7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:53.438561 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e9c30-4941-4cbc-8398-08c6c407c7f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:53.438897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e9c30-4941-4cbc-8398-08c6c407c7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:53.439167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e9c30-4941-4cbc-8398-08c6c407c7f7 admin admin] PATH_INFO: `/v3/projects/0a246906083b4b52a9585d852e14458e` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:53.440437 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e9c30-4941-4cbc-8398-08c6c407c7f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a246906083b4b52a9585d852e14458e)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:53.442442 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e9c30-4941-4cbc-8398-08c6c407c7f7 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:53.559759 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-827e9c30-4941-4cbc-8398-08c6c407c7f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a246906083b4b52a9585d852e14458e', 'request_id': 'req-827e9c30-4941-4cbc-8398-08c6c407c7f7'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:53.636008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-827e9c30-4941-4cbc-8398-08c6c407c7f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a246906083b4b52a9585d852e14458e', 'request_id': 'req-827e9c30-4941-4cbc-8398-08c6c407c7f7'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:53.647950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-827e9c30-4941-4cbc-8398-08c6c407c7f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a246906083b4b52a9585d852e14458e', 'request_id': 'req-827e9c30-4941-4cbc-8398-08c6c407c7f7'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:53.662699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-827e9c30-4941-4cbc-8398-08c6c407c7f7 admin admin] The token cache is being invalidate because project 0a246906083b4b52a9585d852e14458e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:53.663141 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-827e9c30-4941-4cbc-8398-08c6c407c7f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:53.664031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-827e9c30-4941-4cbc-8398-08c6c407c7f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:53.672606 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1021/2077] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:53 2022] DELETE /identity/v3/projects/0a246906083b4b52a9585d852e14458e => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:53.676704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96d319-a815-4259-b60c-dfbf4811d7c1 service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:53.689006 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96d319-a815-4259-b60c-dfbf4811d7c1 service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:53.748086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b35d07-dfc8-4a23-b461-f64e29e94ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:53.748994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b35d07-dfc8-4a23-b461-f64e29e94ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:53.749283 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b35d07-dfc8-4a23-b461-f64e29e94ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:53.749566 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b35d07-dfc8-4a23-b461-f64e29e94ecb admin admin] PATH_INFO: `/v3/users/fede9805b0fa4948a3af31e676aae695` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:53.750761 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b35d07-dfc8-4a23-b461-f64e29e94ecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fede9805b0fa4948a3af31e676aae695)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:53.752844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b35d07-dfc8-4a23-b461-f64e29e94ecb admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:53.976242 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-62b35d07-dfc8-4a23-b461-f64e29e94ecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fede9805b0fa4948a3af31e676aae695'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:53.991399 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-62b35d07-dfc8-4a23-b461-f64e29e94ecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fede9805b0fa4948a3af31e676aae695'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:53.993158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-62b35d07-dfc8-4a23-b461-f64e29e94ecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fede9805b0fa4948a3af31e676aae695'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:53.993802 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-62b35d07-dfc8-4a23-b461-f64e29e94ecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fede9805b0fa4948a3af31e676aae695'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:54.001951 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-62b35d07-dfc8-4a23-b461-f64e29e94ecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fede9805b0fa4948a3af31e676aae695'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:54.109178 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1057/2078] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:53 2022] DELETE /identity/v3/users/fede9805b0fa4948a3af31e676aae695 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:54.297160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e9c30-4941-4cbc-8398-08c6c407c7f7 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:54.300838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-827e9c30-4941-4cbc-8398-08c6c407c7f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:54.311875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e9c30-4941-4cbc-8398-08c6c407c7f7 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:54.383399 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed315eff-fc43-4a48-ac35-45a731a1e5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:54.387312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed315eff-fc43-4a48-ac35-45a731a1e5ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:54.387760 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed315eff-fc43-4a48-ac35-45a731a1e5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:54.387979 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed315eff-fc43-4a48-ac35-45a731a1e5ef admin admin] PATH_INFO: `/v3/users/9d375c5d5e2e400c8b95ffdf39433faf` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:54.389364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed315eff-fc43-4a48-ac35-45a731a1e5ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d375c5d5e2e400c8b95ffdf39433faf)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:54.391596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed315eff-fc43-4a48-ac35-45a731a1e5ef admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:54.473897 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ac177-82c9-46d3-9bfd-2dfc54969422 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:54.474271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ac177-82c9-46d3-9bfd-2dfc54969422 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:54.474633 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ac177-82c9-46d3-9bfd-2dfc54969422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:54.486560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:54.531299 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-462ac177-82c9-46d3-9bfd-2dfc54969422 None None] MFA Rules not processed for user `dd98b2a31e4c46599269684782915703`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:54.533395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-462ac177-82c9-46d3-9bfd-2dfc54969422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:54.633747 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1058/2079] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:46:54.655147 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ac177-82c9-46d3-9bfd-2dfc54969422 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:54.661587 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-462ac177-82c9-46d3-9bfd-2dfc54969422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:54.667736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ed315eff-fc43-4a48-ac35-45a731a1e5ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d375c5d5e2e400c8b95ffdf39433faf'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:54.677128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:54.692066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ed315eff-fc43-4a48-ac35-45a731a1e5ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d375c5d5e2e400c8b95ffdf39433faf'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:54.733150 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ac177-82c9-46d3-9bfd-2dfc54969422 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:54.736994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4eab0-f308-450e-af04-b3bdbddf13c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:54.737590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4eab0-f308-450e-af04-b3bdbddf13c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:54.737881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4eab0-f308-450e-af04-b3bdbddf13c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:54.738166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4eab0-f308-450e-af04-b3bdbddf13c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:54.747735 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4eab0-f308-450e-af04-b3bdbddf13c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:54.749214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4eab0-f308-450e-af04-b3bdbddf13c9 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:54.761250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1059/2080] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:54.761250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4eab0-f308-450e-af04-b3bdbddf13c9 service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:54.765418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-38d4eab0-f308-450e-af04-b3bdbddf13c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:54.776803 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4eab0-f308-450e-af04-b3bdbddf13c9 service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:54.781840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f183bd-7114-403d-a096-a534170c6d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:54.782706 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f183bd-7114-403d-a096-a534170c6d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:54.782997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f183bd-7114-403d-a096-a534170c6d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:54.783278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f183bd-7114-403d-a096-a534170c6d65 admin admin] PATH_INFO: `/v3/projects/cad5f4f6de7841368ef0b01da91fa55d` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:54.784491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f183bd-7114-403d-a096-a534170c6d65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cad5f4f6de7841368ef0b01da91fa55d)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:54.786644 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f183bd-7114-403d-a096-a534170c6d65 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:54.793218 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ed315eff-fc43-4a48-ac35-45a731a1e5ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d375c5d5e2e400c8b95ffdf39433faf'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:54.793218 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ed315eff-fc43-4a48-ac35-45a731a1e5ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d375c5d5e2e400c8b95ffdf39433faf'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:54.793218 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ed315eff-fc43-4a48-ac35-45a731a1e5ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d375c5d5e2e400c8b95ffdf39433faf'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:54.811345 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1022/2081] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:54 2022] DELETE /identity/v3/users/9d375c5d5e2e400c8b95ffdf39433faf => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:54.819792 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed315eff-fc43-4a48-ac35-45a731a1e5ef admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:54.819792 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ed315eff-fc43-4a48-ac35-45a731a1e5ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:54.871904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed315eff-fc43-4a48-ac35-45a731a1e5ef admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:54.875530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d7516-2fc5-4045-9a87-250a4c2d5cca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:54.876173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d7516-2fc5-4045-9a87-250a4c2d5cca service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:54.876459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d7516-2fc5-4045-9a87-250a4c2d5cca service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:54.876773 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d7516-2fc5-4045-9a87-250a4c2d5cca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:54.878572 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e29d7516-2fc5-4045-9a87-250a4c2d5cca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:54.939463 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d7516-2fc5-4045-9a87-250a4c2d5cca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:54.940545 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d7516-2fc5-4045-9a87-250a4c2d5cca service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:54.949396 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a3f183bd-7114-403d-a096-a534170c6d65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cad5f4f6de7841368ef0b01da91fa55d', 'request_id': 'req-a3f183bd-7114-403d-a096-a534170c6d65'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:54.950016 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1023/2082] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:54.952440 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62477174-e52c-48d3-9fde-a137b4b89e0e None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:54.952896 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62477174-e52c-48d3-9fde-a137b4b89e0e None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:54.953365 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62477174-e52c-48d3-9fde-a137b4b89e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:54.956312 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a3f183bd-7114-403d-a096-a534170c6d65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cad5f4f6de7841368ef0b01da91fa55d', 'request_id': 'req-a3f183bd-7114-403d-a096-a534170c6d65'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:54.968321 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a3f183bd-7114-403d-a096-a534170c6d65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cad5f4f6de7841368ef0b01da91fa55d', 'request_id': 'req-a3f183bd-7114-403d-a096-a534170c6d65'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:54.973340 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-62477174-e52c-48d3-9fde-a137b4b89e0e None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:54.974560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-62477174-e52c-48d3-9fde-a137b4b89e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:55.034477 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a3f183bd-7114-403d-a096-a534170c6d65 admin admin] The token cache is being invalidate because project cad5f4f6de7841368ef0b01da91fa55d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:55.035145 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a3f183bd-7114-403d-a096-a534170c6d65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:55.035923 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a3f183bd-7114-403d-a096-a534170c6d65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:55.038052 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1060/2083] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:54 2022] DELETE /identity/v3/projects/cad5f4f6de7841368ef0b01da91fa55d => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:55.043520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806ca6c-7468-4202-8d93-bf48981d8964 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.043722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1024/2084] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:55.043871 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806ca6c-7468-4202-8d93-bf48981d8964 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.044423 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806ca6c-7468-4202-8d93-bf48981d8964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.053534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62477174-e52c-48d3-9fde-a137b4b89e0e None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:55.055550 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-62477174-e52c-48d3-9fde-a137b4b89e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:55.064223 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-b806ca6c-7468-4202-8d93-bf48981d8964 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:55.074546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-b806ca6c-7468-4202-8d93-bf48981d8964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:55.074546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1061/2085] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:46:55.082043 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b95d56-477e-435f-929f-742f942468f2 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.082330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b95d56-477e-435f-929f-742f942468f2 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.082609 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b95d56-477e-435f-929f-742f942468f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.102822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-a8b95d56-477e-435f-929f-742f942468f2 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:55.104026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a8b95d56-477e-435f-929f-742f942468f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:55.114138 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1062/2086] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:46:55.116834 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62477174-e52c-48d3-9fde-a137b4b89e0e None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:55.118801 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b95d56-477e-435f-929f-742f942468f2 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:55.121513 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b54904-b28c-462d-87c9-ff1a3ec5912a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:55.122273 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b54904-b28c-462d-87c9-ff1a3ec5912a service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.122558 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b54904-b28c-462d-87c9-ff1a3ec5912a service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.122826 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b54904-b28c-462d-87c9-ff1a3ec5912a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.125174 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-27b54904-b28c-462d-87c9-ff1a3ec5912a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:55.129579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b95d56-477e-435f-929f-742f942468f2 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:55.134319 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27ae48-f213-4a5d-a139-e0837a1d443d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:55.135114 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27ae48-f213-4a5d-a139-e0837a1d443d admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.135297 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27ae48-f213-4a5d-a139-e0837a1d443d admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.135541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27ae48-f213-4a5d-a139-e0837a1d443d admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.136438 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27ae48-f213-4a5d-a139-e0837a1d443d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:46:55.136772 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27ae48-f213-4a5d-a139-e0837a1d443d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:55.137723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b54904-b28c-462d-87c9-ff1a3ec5912a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:55.138585 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27ae48-f213-4a5d-a139-e0837a1d443d admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:55.139159 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b54904-b28c-462d-87c9-ff1a3ec5912a service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:55.149784 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1025/2087] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:46:55.150960 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1063/2088] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:46:55.155187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b54904-b28c-462d-87c9-ff1a3ec5912a service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:55.165272 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b54904-b28c-462d-87c9-ff1a3ec5912a service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:55.170358 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb4c0e-623d-401c-b5b5-c885cd2c686d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:55.170725 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb4c0e-623d-401c-b5b5-c885cd2c686d admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.171046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb4c0e-623d-401c-b5b5-c885cd2c686d admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.171385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb4c0e-623d-401c-b5b5-c885cd2c686d admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.172024 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb4c0e-623d-401c-b5b5-c885cd2c686d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:55.173927 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb4c0e-623d-401c-b5b5-c885cd2c686d admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:55.298148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1026/2089] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:55 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:46:55.302662 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27ae48-f213-4a5d-a139-e0837a1d443d admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:55.313947 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27ae48-f213-4a5d-a139-e0837a1d443d admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:55.376333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387a779-5ca5-4929-8fa7-e9d9550feb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:55.377143 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387a779-5ca5-4929-8fa7-e9d9550feb90 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.377503 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387a779-5ca5-4929-8fa7-e9d9550feb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.377857 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387a779-5ca5-4929-8fa7-e9d9550feb90 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.378974 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387a779-5ca5-4929-8fa7-e9d9550feb90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:55.380603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387a779-5ca5-4929-8fa7-e9d9550feb90 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:55.518169 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1064/2090] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:55 2022] POST /identity/v3/users => generated 288 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:55.518263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb4c0e-623d-401c-b5b5-c885cd2c686d admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:55.542024 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb4c0e-623d-401c-b5b5-c885cd2c686d admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:55.550623 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea623a-3f9c-4fde-8b21-3b8a3578eee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:55.551309 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea623a-3f9c-4fde-8b21-3b8a3578eee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.551598 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea623a-3f9c-4fde-8b21-3b8a3578eee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.551880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea623a-3f9c-4fde-8b21-3b8a3578eee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.552732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea623a-3f9c-4fde-8b21-3b8a3578eee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:55.554338 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea623a-3f9c-4fde-8b21-3b8a3578eee0 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:55.565852 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1027/2091] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:55.569870 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387a779-5ca5-4929-8fa7-e9d9550feb90 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:55.581011 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387a779-5ca5-4929-8fa7-e9d9550feb90 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:55.586848 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb249c84-53b1-44e8-bcf1-74fd41e25a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:55.587526 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb249c84-53b1-44e8-bcf1-74fd41e25a5c admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.587810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb249c84-53b1-44e8-bcf1-74fd41e25a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.588081 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb249c84-53b1-44e8-bcf1-74fd41e25a5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.588844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb249c84-53b1-44e8-bcf1-74fd41e25a5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:55.590449 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb249c84-53b1-44e8-bcf1-74fd41e25a5c admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:55.600830 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1065/2092] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:55.605525 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea623a-3f9c-4fde-8b21-3b8a3578eee0 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:55.622058 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea623a-3f9c-4fde-8b21-3b8a3578eee0 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:55.624687 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb249c84-53b1-44e8-bcf1-74fd41e25a5c admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:55.627209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-bb249c84-53b1-44e8-bcf1-74fd41e25a5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:55.628724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242215ff-0a8c-41dc-bbb2-88efe3e05b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:55.629865 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242215ff-0a8c-41dc-bbb2-88efe3e05b4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.630358 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242215ff-0a8c-41dc-bbb2-88efe3e05b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.630708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242215ff-0a8c-41dc-bbb2-88efe3e05b4f admin admin] PATH_INFO: `/v3/projects/566af4a530e444438e5353a805501741/users/ad5b8ef100d441c6a978d2b5f0faef71/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.651752 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb249c84-53b1-44e8-bcf1-74fd41e25a5c admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:55.658006 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba306806-fe3d-43cc-bd43-3155d9d33f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:55.659137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba306806-fe3d-43cc-bd43-3155d9d33f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.660230 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba306806-fe3d-43cc-bd43-3155d9d33f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.660729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba306806-fe3d-43cc-bd43-3155d9d33f97 admin admin] PATH_INFO: `/v3/projects/c0492357de44424887f2613db6868b8a` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.662205 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba306806-fe3d-43cc-bd43-3155d9d33f97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0492357de44424887f2613db6868b8a)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:55.664641 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba306806-fe3d-43cc-bd43-3155d9d33f97 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:55.685414 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242215ff-0a8c-41dc-bbb2-88efe3e05b4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=566af4a530e444438e5353a805501741, user_id=ad5b8ef100d441c6a978d2b5f0faef71, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:55.688363 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242215ff-0a8c-41dc-bbb2-88efe3e05b4f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:55.756248 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:55.767804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1028/2093] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:55 2022] PUT /identity/v3/projects/566af4a530e444438e5353a805501741/users/ad5b8ef100d441c6a978d2b5f0faef71/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:55.780724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f257e39-c41a-4320-91ee-c53a5381ad50 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.783426 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f257e39-c41a-4320-91ee-c53a5381ad50 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.783426 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f257e39-c41a-4320-91ee-c53a5381ad50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.783621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-ba306806-fe3d-43cc-bd43-3155d9d33f97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0492357de44424887f2613db6868b8a', 'request_id': 'req-ba306806-fe3d-43cc-bd43-3155d9d33f97'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:55.790574 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-ba306806-fe3d-43cc-bd43-3155d9d33f97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0492357de44424887f2613db6868b8a', 'request_id': 'req-ba306806-fe3d-43cc-bd43-3155d9d33f97'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:55.797320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-ba306806-fe3d-43cc-bd43-3155d9d33f97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0492357de44424887f2613db6868b8a', 'request_id': 'req-ba306806-fe3d-43cc-bd43-3155d9d33f97'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:55.814861 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-1f257e39-c41a-4320-91ee-c53a5381ad50 None None] MFA Rules not processed for user `ad5b8ef100d441c6a978d2b5f0faef71`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:55.816220 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-1f257e39-c41a-4320-91ee-c53a5381ad50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:55.888459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1029/2094] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:46:55.897842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-ba306806-fe3d-43cc-bd43-3155d9d33f97 admin admin] The token cache is being invalidate because project c0492357de44424887f2613db6868b8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:55.897842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-ba306806-fe3d-43cc-bd43-3155d9d33f97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:55.897842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-ba306806-fe3d-43cc-bd43-3155d9d33f97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:55.897842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1066/2095] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:55 2022] DELETE /identity/v3/projects/c0492357de44424887f2613db6868b8a => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:46:55.900478 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f257e39-c41a-4320-91ee-c53a5381ad50 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:55.901646 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba306806-fe3d-43cc-bd43-3155d9d33f97 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:55.902282 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-1f257e39-c41a-4320-91ee-c53a5381ad50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:55.902972 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ba306806-fe3d-43cc-bd43-3155d9d33f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:55.965807 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f257e39-c41a-4320-91ee-c53a5381ad50 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:55.969573 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27859c5-c73e-4a43-ad70-60531620538c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '983801bb18924fe4b3150693765eb13b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:55.972129 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27859c5-c73e-4a43-ad70-60531620538c service glance] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.972129 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27859c5-c73e-4a43-ad70-60531620538c service glance] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.972129 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27859c5-c73e-4a43-ad70-60531620538c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.973333 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c27859c5-c73e-4a43-ad70-60531620538c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:55.974151 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba306806-fe3d-43cc-bd43-3155d9d33f97 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:55.982229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c3d25-0ce8-479a-a0d4-ca721b3d9e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:55.983225 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c3d25-0ce8-479a-a0d4-ca721b3d9e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.983523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c3d25-0ce8-479a-a0d4-ca721b3d9e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.983809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c3d25-0ce8-479a-a0d4-ca721b3d9e88 admin admin] PATH_INFO: `/v3/users/42d52b44c7aa4d1c843780e9d47dc5f6` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.985305 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c3d25-0ce8-479a-a0d4-ca721b3d9e88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42d52b44c7aa4d1c843780e9d47dc5f6)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:55.987713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c3d25-0ce8-479a-a0d4-ca721b3d9e88 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:55.988044 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27859c5-c73e-4a43-ad70-60531620538c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:55.989738 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27859c5-c73e-4a43-ad70-60531620538c service glance] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:56.001959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1030/2096] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:56.259834 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2e8c3d25-0ce8-479a-a0d4-ca721b3d9e88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42d52b44c7aa4d1c843780e9d47dc5f6'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:56.282163 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2e8c3d25-0ce8-479a-a0d4-ca721b3d9e88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42d52b44c7aa4d1c843780e9d47dc5f6'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:56.283079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2e8c3d25-0ce8-479a-a0d4-ca721b3d9e88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42d52b44c7aa4d1c843780e9d47dc5f6'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:56.283866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2e8c3d25-0ce8-479a-a0d4-ca721b3d9e88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42d52b44c7aa4d1c843780e9d47dc5f6'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:56.293551 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2e8c3d25-0ce8-479a-a0d4-ca721b3d9e88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42d52b44c7aa4d1c843780e9d47dc5f6'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:56.371868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1067/2097] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:55 2022] DELETE /identity/v3/users/42d52b44c7aa4d1c843780e9d47dc5f6 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:46:56.661604 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8f62d-7962-4ee9-a26e-18caf33e87f8 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:56.662134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8f62d-7962-4ee9-a26e-18caf33e87f8 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:56.662253 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8f62d-7962-4ee9-a26e-18caf33e87f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:56.687811 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-77d8f62d-7962-4ee9-a26e-18caf33e87f8 None None] MFA Rules not processed for user `dd98b2a31e4c46599269684782915703`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:56.689342 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-77d8f62d-7962-4ee9-a26e-18caf33e87f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:56.745077 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:56.761026 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1031/2098] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:56.929704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c3d25-0ce8-479a-a0d4-ca721b3d9e88 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:56.931269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-2e8c3d25-0ce8-479a-a0d4-ca721b3d9e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:57.004635 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c3d25-0ce8-479a-a0d4-ca721b3d9e88 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:57.010805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c40bdd-c6c2-4156-b52f-1c596e4378d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:57.011766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c40bdd-c6c2-4156-b52f-1c596e4378d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:57.012172 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c40bdd-c6c2-4156-b52f-1c596e4378d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:57.012516 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c40bdd-c6c2-4156-b52f-1c596e4378d5 admin admin] PATH_INFO: `/v3/projects/ed48a88e673e4716ae7a65f5b8318a2b` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:57.013795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c40bdd-c6c2-4156-b52f-1c596e4378d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed48a88e673e4716ae7a65f5b8318a2b)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:57.015731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c40bdd-c6c2-4156-b52f-1c596e4378d5 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:57.134159 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b4c40bdd-c6c2-4156-b52f-1c596e4378d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed48a88e673e4716ae7a65f5b8318a2b', 'request_id': 'req-b4c40bdd-c6c2-4156-b52f-1c596e4378d5'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:57.140256 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b4c40bdd-c6c2-4156-b52f-1c596e4378d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed48a88e673e4716ae7a65f5b8318a2b', 'request_id': 'req-b4c40bdd-c6c2-4156-b52f-1c596e4378d5'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:57.146022 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b4c40bdd-c6c2-4156-b52f-1c596e4378d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed48a88e673e4716ae7a65f5b8318a2b', 'request_id': 'req-b4c40bdd-c6c2-4156-b52f-1c596e4378d5'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:57.171553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:57.240068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b4c40bdd-c6c2-4156-b52f-1c596e4378d5 admin admin] The token cache is being invalidate because project ed48a88e673e4716ae7a65f5b8318a2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:57.240524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b4c40bdd-c6c2-4156-b52f-1c596e4378d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:57.241205 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b4c40bdd-c6c2-4156-b52f-1c596e4378d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:57.243170 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1068/2099] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:56 2022] DELETE /identity/v3/projects/ed48a88e673e4716ae7a65f5b8318a2b => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:46:57.248543 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1d41b-e323-4e3b-bb5c-c9e77bf10038 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:57.248861 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1d41b-e323-4e3b-bb5c-c9e77bf10038 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:57.249646 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1d41b-e323-4e3b-bb5c-c9e77bf10038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:57.300448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-7ec1d41b-e323-4e3b-bb5c-c9e77bf10038 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:57.306428 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-7ec1d41b-e323-4e3b-bb5c-c9e77bf10038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:57.367303 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1032/2100] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:57.376288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84198135-a328-4495-b843-a36241eba907 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:57.376600 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84198135-a328-4495-b843-a36241eba907 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:57.376905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84198135-a328-4495-b843-a36241eba907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:57.396097 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-84198135-a328-4495-b843-a36241eba907 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:57.397766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-84198135-a328-4495-b843-a36241eba907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:57.407398 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1069/2101] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:57.411557 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1d41b-e323-4e3b-bb5c-c9e77bf10038 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:57.423458 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1d41b-e323-4e3b-bb5c-c9e77bf10038 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:57.429929 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33335c-9376-42c7-87d1-cf24d4e94780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:57.431207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33335c-9376-42c7-87d1-cf24d4e94780 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:57.431793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33335c-9376-42c7-87d1-cf24d4e94780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:57.432221 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33335c-9376-42c7-87d1-cf24d4e94780 admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:57.433272 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33335c-9376-42c7-87d1-cf24d4e94780 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:46:57.433827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33335c-9376-42c7-87d1-cf24d4e94780 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:57.435736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33335c-9376-42c7-87d1-cf24d4e94780 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:57.451294 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1033/2102] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:46:57.457146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84198135-a328-4495-b843-a36241eba907 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:57.470366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84198135-a328-4495-b843-a36241eba907 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:57.476398 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f49c7-8b6c-45a8-9792-267fe3f7b652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:57.477377 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f49c7-8b6c-45a8-9792-267fe3f7b652 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:57.477763 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f49c7-8b6c-45a8-9792-267fe3f7b652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:57.478106 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f49c7-8b6c-45a8-9792-267fe3f7b652 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:57.479032 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594f49c7-8b6c-45a8-9792-267fe3f7b652 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:57.481201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594f49c7-8b6c-45a8-9792-267fe3f7b652 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:57.593408 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1070/2103] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:57 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:46:57.594422 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33335c-9376-42c7-87d1-cf24d4e94780 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:57.604976 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33335c-9376-42c7-87d1-cf24d4e94780 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:57.670788 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa67a1-e09c-444c-a307-162890fe4429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:57.671499 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa67a1-e09c-444c-a307-162890fe4429 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:57.671735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa67a1-e09c-444c-a307-162890fe4429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:57.672017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa67a1-e09c-444c-a307-162890fe4429 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:57.672771 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa67a1-e09c-444c-a307-162890fe4429 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:57.674727 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa67a1-e09c-444c-a307-162890fe4429 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:57.803274 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1034/2104] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:57 2022] POST /identity/v3/users => generated 294 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:57.807836 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f49c7-8b6c-45a8-9792-267fe3f7b652 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:57.826726 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f49c7-8b6c-45a8-9792-267fe3f7b652 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:57.832423 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ae836-52ee-422e-b872-b21696982f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:57.833390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ae836-52ee-422e-b872-b21696982f10 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:57.833857 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ae836-52ee-422e-b872-b21696982f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:57.834247 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ae836-52ee-422e-b872-b21696982f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:57.835439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ae836-52ee-422e-b872-b21696982f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:57.837835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ae836-52ee-422e-b872-b21696982f10 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:57.864397 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1071/2105] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:57.875482 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa67a1-e09c-444c-a307-162890fe4429 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:57.883435 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa67a1-e09c-444c-a307-162890fe4429 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:57.896306 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2564cf3-7045-4f8e-928a-9bd0d73972f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:57.897029 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2564cf3-7045-4f8e-928a-9bd0d73972f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:57.897312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2564cf3-7045-4f8e-928a-9bd0d73972f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:57.897591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2564cf3-7045-4f8e-928a-9bd0d73972f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:57.898429 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2564cf3-7045-4f8e-928a-9bd0d73972f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:57.900032 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2564cf3-7045-4f8e-928a-9bd0d73972f5 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:57.912620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1035/2106] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:57.916617 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ae836-52ee-422e-b872-b21696982f10 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:57.931994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ae836-52ee-422e-b872-b21696982f10 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:57.939090 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d46269-9317-4136-bc9c-b14c3753fc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:57.940045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d46269-9317-4136-bc9c-b14c3753fc46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:57.940325 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d46269-9317-4136-bc9c-b14c3753fc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:57.940625 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d46269-9317-4136-bc9c-b14c3753fc46 admin admin] PATH_INFO: `/v3/projects/4ff9aee2da424343b2f42c313d4116db/users/78b8b5f7cf944ff1a9b6396bf0072a28/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:57.956733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d46269-9317-4136-bc9c-b14c3753fc46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ff9aee2da424343b2f42c313d4116db, user_id=78b8b5f7cf944ff1a9b6396bf0072a28, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:57.959579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d46269-9317-4136-bc9c-b14c3753fc46 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:58.060416 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1072/2107] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:57 2022] PUT /identity/v3/projects/4ff9aee2da424343b2f42c313d4116db/users/78b8b5f7cf944ff1a9b6396bf0072a28/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:58.061213 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471448d-f510-4d43-aecb-b63c11364a9c None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:58.061213 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471448d-f510-4d43-aecb-b63c11364a9c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:58.061213 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471448d-f510-4d43-aecb-b63c11364a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:58.088285 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-2471448d-f510-4d43-aecb-b63c11364a9c None None] MFA Rules not processed for user `78b8b5f7cf944ff1a9b6396bf0072a28`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:58.089576 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-2471448d-f510-4d43-aecb-b63c11364a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:58.158692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1036/2108] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:58.163203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d46269-9317-4136-bc9c-b14c3753fc46 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:58.237223 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d46269-9317-4136-bc9c-b14c3753fc46 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:58.242257 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4339b89-237b-4daf-8013-cae60a8dc22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:58.242992 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4339b89-237b-4daf-8013-cae60a8dc22c admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:58.243489 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4339b89-237b-4daf-8013-cae60a8dc22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:58.243586 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4339b89-237b-4daf-8013-cae60a8dc22c admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:58.244390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4339b89-237b-4daf-8013-cae60a8dc22c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:58.246585 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4339b89-237b-4daf-8013-cae60a8dc22c admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:58.263161 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:58.324619 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1073/2109] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:58 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:58.328723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471448d-f510-4d43-aecb-b63c11364a9c None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:58.340702 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471448d-f510-4d43-aecb-b63c11364a9c None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:58.404271 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2a091-e021-482a-89c8-da0427438582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:58.405219 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2a091-e021-482a-89c8-da0427438582 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:58.405686 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2a091-e021-482a-89c8-da0427438582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:58.405968 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2a091-e021-482a-89c8-da0427438582 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:58.406678 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2a091-e021-482a-89c8-da0427438582 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:58.408672 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2a091-e021-482a-89c8-da0427438582 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:58.472802 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1037/2110] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:58 2022] POST /identity/v3/users => generated 300 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:58.474101 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4339b89-237b-4daf-8013-cae60a8dc22c admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:58.485660 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4339b89-237b-4daf-8013-cae60a8dc22c admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:58.490710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252235c-9a2c-46c5-a9f6-d26be2ba3c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:58.491381 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252235c-9a2c-46c5-a9f6-d26be2ba3c45 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:58.491657 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252235c-9a2c-46c5-a9f6-d26be2ba3c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:58.492036 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252235c-9a2c-46c5-a9f6-d26be2ba3c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:58.493095 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252235c-9a2c-46c5-a9f6-d26be2ba3c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:58.495171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252235c-9a2c-46c5-a9f6-d26be2ba3c45 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:58.508619 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1074/2111] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:58.513836 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2a091-e021-482a-89c8-da0427438582 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:58.525915 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2a091-e021-482a-89c8-da0427438582 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:58.531515 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8436fb3-98d2-4820-bc9a-2629e2b72f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:58.533581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8436fb3-98d2-4820-bc9a-2629e2b72f58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:58.533581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8436fb3-98d2-4820-bc9a-2629e2b72f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:58.533581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8436fb3-98d2-4820-bc9a-2629e2b72f58 admin admin] PATH_INFO: `/v3/projects/1d4ee33ff98c403a8c27923ebc9b1b03/users/f5202eb7618746739907e95a2a6e56d0/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:58.557640 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8436fb3-98d2-4820-bc9a-2629e2b72f58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d4ee33ff98c403a8c27923ebc9b1b03, user_id=f5202eb7618746739907e95a2a6e56d0, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:58.560567 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8436fb3-98d2-4820-bc9a-2629e2b72f58 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:58.669866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1038/2112] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:58 2022] PUT /identity/v3/projects/1d4ee33ff98c403a8c27923ebc9b1b03/users/f5202eb7618746739907e95a2a6e56d0/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:58.670000 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252235c-9a2c-46c5-a9f6-d26be2ba3c45 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:58.680711 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252235c-9a2c-46c5-a9f6-d26be2ba3c45 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:58.752488 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83dfea8-e7b7-4107-91ba-5092fa5e2262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:58.753213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dfea8-e7b7-4107-91ba-5092fa5e2262 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:58.753506 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dfea8-e7b7-4107-91ba-5092fa5e2262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:58.753790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dfea8-e7b7-4107-91ba-5092fa5e2262 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:58.754692 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83dfea8-e7b7-4107-91ba-5092fa5e2262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:58.756415 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83dfea8-e7b7-4107-91ba-5092fa5e2262 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:58.768994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1075/2113] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:58.774194 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8436fb3-98d2-4820-bc9a-2629e2b72f58 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:58.789113 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8436fb3-98d2-4820-bc9a-2629e2b72f58 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:58.796857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835714b1-6e24-48e0-90c3-ca701c35bd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:58.798047 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835714b1-6e24-48e0-90c3-ca701c35bd3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:58.798334 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835714b1-6e24-48e0-90c3-ca701c35bd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:58.798611 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835714b1-6e24-48e0-90c3-ca701c35bd3c admin admin] PATH_INFO: `/v3/projects/1d4ee33ff98c403a8c27923ebc9b1b03/users/f5202eb7618746739907e95a2a6e56d0/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:58.800724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835714b1-6e24-48e0-90c3-ca701c35bd3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d4ee33ff98c403a8c27923ebc9b1b03, user_id=f5202eb7618746739907e95a2a6e56d0, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:58.803608 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835714b1-6e24-48e0-90c3-ca701c35bd3c admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:58.814698 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1039/2114] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:58 2022] PUT /identity/v3/projects/1d4ee33ff98c403a8c27923ebc9b1b03/users/f5202eb7618746739907e95a2a6e56d0/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:58.826405 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415420bd-374c-44da-8927-a33d087cc0d0 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:58.826699 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415420bd-374c-44da-8927-a33d087cc0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:58.826983 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415420bd-374c-44da-8927-a33d087cc0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:58.856942 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-415420bd-374c-44da-8927-a33d087cc0d0 None None] MFA Rules not processed for user `f5202eb7618746739907e95a2a6e56d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:58.858203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-415420bd-374c-44da-8927-a33d087cc0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:58.946351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1076/2115] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:58.960064 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:58.970795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835714b1-6e24-48e0-90c3-ca701c35bd3c admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:58.972618 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-835714b1-6e24-48e0-90c3-ca701c35bd3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:59.032639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835714b1-6e24-48e0-90c3-ca701c35bd3c admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:59.037052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb40d9-3d58-4e9d-b07c-d04318751240 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:59.037974 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb40d9-3d58-4e9d-b07c-d04318751240 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:59.038830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb40d9-3d58-4e9d-b07c-d04318751240 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:59.038830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb40d9-3d58-4e9d-b07c-d04318751240 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:59.049387 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb40d9-3d58-4e9d-b07c-d04318751240 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:59.050833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb40d9-3d58-4e9d-b07c-d04318751240 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:59.060899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1040/2116] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:59.126282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415420bd-374c-44da-8927-a33d087cc0d0 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:59.189916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415420bd-374c-44da-8927-a33d087cc0d0 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:59.194757 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870914d4-cbb6-474f-8d0a-570de6455ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:59.196408 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870914d4-cbb6-474f-8d0a-570de6455ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:59.196725 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870914d4-cbb6-474f-8d0a-570de6455ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:59.197018 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870914d4-cbb6-474f-8d0a-570de6455ffd admin admin] PATH_INFO: `/v3/users/78b8b5f7cf944ff1a9b6396bf0072a28` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:59.198345 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870914d4-cbb6-474f-8d0a-570de6455ffd admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b8b5f7cf944ff1a9b6396bf0072a28)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:59.200281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870914d4-cbb6-474f-8d0a-570de6455ffd admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:59.386946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-870914d4-cbb6-474f-8d0a-570de6455ffd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b8b5f7cf944ff1a9b6396bf0072a28'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:59.401513 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:59.402974 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-870914d4-cbb6-474f-8d0a-570de6455ffd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b8b5f7cf944ff1a9b6396bf0072a28'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:59.403883 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-870914d4-cbb6-474f-8d0a-570de6455ffd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b8b5f7cf944ff1a9b6396bf0072a28'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:59.404606 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-870914d4-cbb6-474f-8d0a-570de6455ffd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b8b5f7cf944ff1a9b6396bf0072a28'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:59.413450 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-870914d4-cbb6-474f-8d0a-570de6455ffd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b8b5f7cf944ff1a9b6396bf0072a28'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:59.500160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1077/2117] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:59 2022] DELETE /identity/v3/users/78b8b5f7cf944ff1a9b6396bf0072a28 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:59.504840 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e981ead-cbf9-4635-bb92-5339afde6096 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:59.505132 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e981ead-cbf9-4635-bb92-5339afde6096 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:59.505414 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e981ead-cbf9-4635-bb92-5339afde6096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:59.524571 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-2e981ead-cbf9-4635-bb92-5339afde6096 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:59.525938 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-2e981ead-cbf9-4635-bb92-5339afde6096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:59.555896 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870914d4-cbb6-474f-8d0a-570de6455ffd admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:59.557606 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-870914d4-cbb6-474f-8d0a-570de6455ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:59.589816 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1041/2118] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:59.599741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e981ead-cbf9-4635-bb92-5339afde6096 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:59.601800 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-2e981ead-cbf9-4635-bb92-5339afde6096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:59.627362 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870914d4-cbb6-474f-8d0a-570de6455ffd admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:59.637059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1048dfd-b5ad-47e2-9f4d-664e8c58408c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:59.638129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1048dfd-b5ad-47e2-9f4d-664e8c58408c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:59.638470 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1048dfd-b5ad-47e2-9f4d-664e8c58408c admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:59.638862 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1048dfd-b5ad-47e2-9f4d-664e8c58408c admin admin] PATH_INFO: `/v3/users/3f08821872cb4db389110a1efac1d6a7` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:59.640294 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1048dfd-b5ad-47e2-9f4d-664e8c58408c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f08821872cb4db389110a1efac1d6a7)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:59.643508 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1048dfd-b5ad-47e2-9f4d-664e8c58408c admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:59.660752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e981ead-cbf9-4635-bb92-5339afde6096 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:59.665503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209a99d-b136-43d8-b2bf-6031d97e0d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:59.666358 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209a99d-b136-43d8-b2bf-6031d97e0d66 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:59.666879 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209a99d-b136-43d8-b2bf-6031d97e0d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:59.667173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209a99d-b136-43d8-b2bf-6031d97e0d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:59.677360 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5209a99d-b136-43d8-b2bf-6031d97e0d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:59.678936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5209a99d-b136-43d8-b2bf-6031d97e0d66 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:59.689731 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1042/2119] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:46:59.760113 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:59.832214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d1048dfd-b5ad-47e2-9f4d-664e8c58408c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f08821872cb4db389110a1efac1d6a7'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:59.854448 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:59.856332 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d1048dfd-b5ad-47e2-9f4d-664e8c58408c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f08821872cb4db389110a1efac1d6a7'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:59.857202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d1048dfd-b5ad-47e2-9f4d-664e8c58408c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f08821872cb4db389110a1efac1d6a7'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:59.857891 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d1048dfd-b5ad-47e2-9f4d-664e8c58408c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f08821872cb4db389110a1efac1d6a7'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:59.866127 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d1048dfd-b5ad-47e2-9f4d-664e8c58408c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f08821872cb4db389110a1efac1d6a7'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:59.979709 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1078/2120] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:59 2022] DELETE /identity/v3/users/3f08821872cb4db389110a1efac1d6a7 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:46:59.986566 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddeec4b-0a1d-4a35-99d6-bbcd86ffde5a None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:59.987485 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddeec4b-0a1d-4a35-99d6-bbcd86ffde5a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:59.988196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddeec4b-0a1d-4a35-99d6-bbcd86ffde5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:00.012716 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-3ddeec4b-0a1d-4a35-99d6-bbcd86ffde5a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:00.012716 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3ddeec4b-0a1d-4a35-99d6-bbcd86ffde5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:00.084530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1043/2121] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:46:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:00.095201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1048dfd-b5ad-47e2-9f4d-664e8c58408c admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:00.098772 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d1048dfd-b5ad-47e2-9f4d-664e8c58408c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:00.167557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1048dfd-b5ad-47e2-9f4d-664e8c58408c admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:00.172288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d532740-3478-4fd1-88e9-e0bfde995dca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:00.172987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d532740-3478-4fd1-88e9-e0bfde995dca service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:00.176932 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d532740-3478-4fd1-88e9-e0bfde995dca service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:00.176932 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d532740-3478-4fd1-88e9-e0bfde995dca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:00.197297 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d532740-3478-4fd1-88e9-e0bfde995dca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:00.198947 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d532740-3478-4fd1-88e9-e0bfde995dca service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:00.210174 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1079/2122] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:00.259814 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddeec4b-0a1d-4a35-99d6-bbcd86ffde5a None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:00.263272 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3ddeec4b-0a1d-4a35-99d6-bbcd86ffde5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:00.281687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddeec4b-0a1d-4a35-99d6-bbcd86ffde5a None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:00.297541 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e770687-5e46-480b-9304-ea08e99dad82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:00.298703 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e770687-5e46-480b-9304-ea08e99dad82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:00.299401 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e770687-5e46-480b-9304-ea08e99dad82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:00.299786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e770687-5e46-480b-9304-ea08e99dad82 admin admin] PATH_INFO: `/v3/projects/4ff9aee2da424343b2f42c313d4116db` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:00.301737 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e770687-5e46-480b-9304-ea08e99dad82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ff9aee2da424343b2f42c313d4116db)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:00.305615 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e770687-5e46-480b-9304-ea08e99dad82 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:00.327105 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:00.458078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9e770687-5e46-480b-9304-ea08e99dad82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ff9aee2da424343b2f42c313d4116db', 'request_id': 'req-9e770687-5e46-480b-9304-ea08e99dad82'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:00.554723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9e770687-5e46-480b-9304-ea08e99dad82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ff9aee2da424343b2f42c313d4116db', 'request_id': 'req-9e770687-5e46-480b-9304-ea08e99dad82'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:00.569104 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:00.575117 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9e770687-5e46-480b-9304-ea08e99dad82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ff9aee2da424343b2f42c313d4116db', 'request_id': 'req-9e770687-5e46-480b-9304-ea08e99dad82'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:00.595374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9e770687-5e46-480b-9304-ea08e99dad82 admin admin] The token cache is being invalidate because project 4ff9aee2da424343b2f42c313d4116db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:00.595741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9e770687-5e46-480b-9304-ea08e99dad82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:00.596756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9e770687-5e46-480b-9304-ea08e99dad82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:00.600239 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1044/2123] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:00 2022] DELETE /identity/v3/projects/4ff9aee2da424343b2f42c313d4116db => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:00.607641 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d532740-3478-4fd1-88e9-e0bfde995dca service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:00.620278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d532740-3478-4fd1-88e9-e0bfde995dca service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:00.698198 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:00.700236 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8ea10-2af6-49f6-8563-6e0c574496a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:00.703336 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8ea10-2af6-49f6-8563-6e0c574496a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:00.708418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8ea10-2af6-49f6-8563-6e0c574496a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:00.708418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8ea10-2af6-49f6-8563-6e0c574496a4 admin admin] PATH_INFO: `/v3/users/f5202eb7618746739907e95a2a6e56d0` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:00.708418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8ea10-2af6-49f6-8563-6e0c574496a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5202eb7618746739907e95a2a6e56d0)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:00.708418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8ea10-2af6-49f6-8563-6e0c574496a4 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:00.998956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d6f8ea10-2af6-49f6-8563-6e0c574496a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5202eb7618746739907e95a2a6e56d0'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:01.012304 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d6f8ea10-2af6-49f6-8563-6e0c574496a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5202eb7618746739907e95a2a6e56d0'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:01.013740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d6f8ea10-2af6-49f6-8563-6e0c574496a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5202eb7618746739907e95a2a6e56d0'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:01.017872 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d6f8ea10-2af6-49f6-8563-6e0c574496a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5202eb7618746739907e95a2a6e56d0'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:01.033574 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d6f8ea10-2af6-49f6-8563-6e0c574496a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5202eb7618746739907e95a2a6e56d0'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:01.114588 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e770687-5e46-480b-9304-ea08e99dad82 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:01.116537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-9e770687-5e46-480b-9304-ea08e99dad82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:01.133847 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e770687-5e46-480b-9304-ea08e99dad82 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:01.139559 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40da05-ea97-467f-825c-218957120907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:01.140781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40da05-ea97-467f-825c-218957120907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:01.141546 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40da05-ea97-467f-825c-218957120907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:01.142063 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40da05-ea97-467f-825c-218957120907 admin admin] PATH_INFO: `/v3/projects/6e4755fbd98d4b8cb9b40ddfa178c421` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:01.143526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40da05-ea97-467f-825c-218957120907 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e4755fbd98d4b8cb9b40ddfa178c421)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:01.146457 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40da05-ea97-467f-825c-218957120907 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:01.148547 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1080/2124] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:00 2022] DELETE /identity/v3/users/f5202eb7618746739907e95a2a6e56d0 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:01.286962 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:01.310866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fd40da05-ea97-467f-825c-218957120907 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e4755fbd98d4b8cb9b40ddfa178c421', 'request_id': 'req-fd40da05-ea97-467f-825c-218957120907'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:01.365614 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fd40da05-ea97-467f-825c-218957120907 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e4755fbd98d4b8cb9b40ddfa178c421', 'request_id': 'req-fd40da05-ea97-467f-825c-218957120907'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:01.373903 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fd40da05-ea97-467f-825c-218957120907 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e4755fbd98d4b8cb9b40ddfa178c421', 'request_id': 'req-fd40da05-ea97-467f-825c-218957120907'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:01.385115 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fd40da05-ea97-467f-825c-218957120907 admin admin] The token cache is being invalidate because project 6e4755fbd98d4b8cb9b40ddfa178c421 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:01.385876 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fd40da05-ea97-467f-825c-218957120907 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:01.387019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fd40da05-ea97-467f-825c-218957120907 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:01.391788 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1045/2125] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:01 2022] DELETE /identity/v3/projects/6e4755fbd98d4b8cb9b40ddfa178c421 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:01.393084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8ea10-2af6-49f6-8563-6e0c574496a4 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:01.394706 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d6f8ea10-2af6-49f6-8563-6e0c574496a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:01.474459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8ea10-2af6-49f6-8563-6e0c574496a4 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:01.547397 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504600c-d74f-46e0-801f-c224622827d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:01.553302 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504600c-d74f-46e0-801f-c224622827d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:01.553302 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504600c-d74f-46e0-801f-c224622827d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:01.553302 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504600c-d74f-46e0-801f-c224622827d1 admin admin] PATH_INFO: `/v3/users/5305e6dfae5a443eb08f4ac6cb026cde` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:01.553302 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504600c-d74f-46e0-801f-c224622827d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5305e6dfae5a443eb08f4ac6cb026cde)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:01.553302 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504600c-d74f-46e0-801f-c224622827d1 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:01.762840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-0504600c-d74f-46e0-801f-c224622827d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5305e6dfae5a443eb08f4ac6cb026cde'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:01.775534 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-0504600c-d74f-46e0-801f-c224622827d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5305e6dfae5a443eb08f4ac6cb026cde'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:01.776359 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-0504600c-d74f-46e0-801f-c224622827d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5305e6dfae5a443eb08f4ac6cb026cde'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:01.777055 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-0504600c-d74f-46e0-801f-c224622827d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5305e6dfae5a443eb08f4ac6cb026cde'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:01.784663 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-0504600c-d74f-46e0-801f-c224622827d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5305e6dfae5a443eb08f4ac6cb026cde'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:01.845302 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1081/2126] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:01 2022] DELETE /identity/v3/users/5305e6dfae5a443eb08f4ac6cb026cde => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:01.852080 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:01.910076 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40da05-ea97-467f-825c-218957120907 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:01.910076 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-fd40da05-ea97-467f-825c-218957120907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:01.921981 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40da05-ea97-467f-825c-218957120907 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:02.011849 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6173b7-333e-4291-8fdf-a6f6888833f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:02.013045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6173b7-333e-4291-8fdf-a6f6888833f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:02.013516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6173b7-333e-4291-8fdf-a6f6888833f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:02.013830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6173b7-333e-4291-8fdf-a6f6888833f1 admin admin] PATH_INFO: `/v3/projects/1d4ee33ff98c403a8c27923ebc9b1b03` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:02.016206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6173b7-333e-4291-8fdf-a6f6888833f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d4ee33ff98c403a8c27923ebc9b1b03)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:02.019834 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6173b7-333e-4291-8fdf-a6f6888833f1 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:02.064878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:02.194560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-af6173b7-333e-4291-8fdf-a6f6888833f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d4ee33ff98c403a8c27923ebc9b1b03', 'request_id': 'req-af6173b7-333e-4291-8fdf-a6f6888833f1'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:02.279462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-af6173b7-333e-4291-8fdf-a6f6888833f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d4ee33ff98c403a8c27923ebc9b1b03', 'request_id': 'req-af6173b7-333e-4291-8fdf-a6f6888833f1'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:02.286406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-af6173b7-333e-4291-8fdf-a6f6888833f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d4ee33ff98c403a8c27923ebc9b1b03', 'request_id': 'req-af6173b7-333e-4291-8fdf-a6f6888833f1'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:02.298572 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-af6173b7-333e-4291-8fdf-a6f6888833f1 admin admin] The token cache is being invalidate because project 1d4ee33ff98c403a8c27923ebc9b1b03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:02.298840 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-af6173b7-333e-4291-8fdf-a6f6888833f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:02.299574 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-af6173b7-333e-4291-8fdf-a6f6888833f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:02.301278 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1046/2127] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:01 2022] DELETE /identity/v3/projects/1d4ee33ff98c403a8c27923ebc9b1b03 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:02.306305 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cabc97-9a8c-4adb-ae91-aaeee874b931 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:02.306760 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cabc97-9a8c-4adb-ae91-aaeee874b931 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:02.307198 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cabc97-9a8c-4adb-ae91-aaeee874b931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:02.326262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-59cabc97-9a8c-4adb-ae91-aaeee874b931 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:02.327668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-59cabc97-9a8c-4adb-ae91-aaeee874b931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:02.393765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1082/2128] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:47:02.402490 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05610dad-ac2f-409c-80c8-e2fe4ec632ca None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:02.402837 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05610dad-ac2f-409c-80c8-e2fe4ec632ca None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:02.403123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05610dad-ac2f-409c-80c8-e2fe4ec632ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:02.425705 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-05610dad-ac2f-409c-80c8-e2fe4ec632ca None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:02.427033 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-05610dad-ac2f-409c-80c8-e2fe4ec632ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:02.438932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1047/2129] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:02.440304 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cabc97-9a8c-4adb-ae91-aaeee874b931 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:02.450583 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cabc97-9a8c-4adb-ae91-aaeee874b931 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:02.455615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a88c06-16f7-48d3-ad37-6fba1776e550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:02.456361 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a88c06-16f7-48d3-ad37-6fba1776e550 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:02.456566 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a88c06-16f7-48d3-ad37-6fba1776e550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:02.456965 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a88c06-16f7-48d3-ad37-6fba1776e550 admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:02.457613 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a88c06-16f7-48d3-ad37-6fba1776e550 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:47:02.461572 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a88c06-16f7-48d3-ad37-6fba1776e550 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:02.461572 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a88c06-16f7-48d3-ad37-6fba1776e550 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:02.472348 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1083/2130] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:47:02.477300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05610dad-ac2f-409c-80c8-e2fe4ec632ca None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:02.486919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05610dad-ac2f-409c-80c8-e2fe4ec632ca None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:02.494731 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1025bf-e97d-47f5-9aa2-761ffcae9e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:02.495421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1025bf-e97d-47f5-9aa2-761ffcae9e78 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:02.495553 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a88c06-16f7-48d3-ad37-6fba1776e550 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:02.496089 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1025bf-e97d-47f5-9aa2-761ffcae9e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:02.496546 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1025bf-e97d-47f5-9aa2-761ffcae9e78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:02.497542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-33a88c06-16f7-48d3-ad37-6fba1776e550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:02.497764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1025bf-e97d-47f5-9aa2-761ffcae9e78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:02.500307 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1025bf-e97d-47f5-9aa2-761ffcae9e78 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:02.512283 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a88c06-16f7-48d3-ad37-6fba1776e550 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:02.526496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21224a4-fd1d-4807-bb58-d773a4487099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:02.527408 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21224a4-fd1d-4807-bb58-d773a4487099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:02.527697 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21224a4-fd1d-4807-bb58-d773a4487099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:02.528156 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21224a4-fd1d-4807-bb58-d773a4487099 admin admin] PATH_INFO: `/v3/projects/792adfa3e25441f7955e6b916a9f1ab6` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:02.529626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21224a4-fd1d-4807-bb58-d773a4487099 admin admin] RBAC: Authorizing `identity:delete_project(project_id=792adfa3e25441f7955e6b916a9f1ab6)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:02.532259 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21224a4-fd1d-4807-bb58-d773a4487099 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:02.620055 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1048/2131] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:02 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:02.624173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1025bf-e97d-47f5-9aa2-761ffcae9e78 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:02.634564 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:02.662324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a21224a4-fd1d-4807-bb58-d773a4487099 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '792adfa3e25441f7955e6b916a9f1ab6', 'request_id': 'req-a21224a4-fd1d-4807-bb58-d773a4487099'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:02.672537 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a21224a4-fd1d-4807-bb58-d773a4487099 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '792adfa3e25441f7955e6b916a9f1ab6', 'request_id': 'req-a21224a4-fd1d-4807-bb58-d773a4487099'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:02.679944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a21224a4-fd1d-4807-bb58-d773a4487099 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '792adfa3e25441f7955e6b916a9f1ab6', 'request_id': 'req-a21224a4-fd1d-4807-bb58-d773a4487099'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:02.734231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1025bf-e97d-47f5-9aa2-761ffcae9e78 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:02.750607 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a21224a4-fd1d-4807-bb58-d773a4487099 admin admin] The token cache is being invalidate because project 792adfa3e25441f7955e6b916a9f1ab6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:02.750967 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a21224a4-fd1d-4807-bb58-d773a4487099 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:02.757906 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a21224a4-fd1d-4807-bb58-d773a4487099 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:02.762452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1084/2132] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:02 2022] DELETE /identity/v3/projects/792adfa3e25441f7955e6b916a9f1ab6 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:02.768979 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21224a4-fd1d-4807-bb58-d773a4487099 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:02.775595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a21224a4-fd1d-4807-bb58-d773a4487099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:02.829347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef4de7-1b7d-4565-b3bb-b22823b07834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:02.830155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef4de7-1b7d-4565-b3bb-b22823b07834 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:02.830442 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef4de7-1b7d-4565-b3bb-b22823b07834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:02.830771 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef4de7-1b7d-4565-b3bb-b22823b07834 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:02.831615 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef4de7-1b7d-4565-b3bb-b22823b07834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:02.833612 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef4de7-1b7d-4565-b3bb-b22823b07834 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:02.863403 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21224a4-fd1d-4807-bb58-d773a4487099 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:02.869951 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d435034-4594-4d96-8c4b-bd9ac6b61082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:02.871017 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d435034-4594-4d96-8c4b-bd9ac6b61082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:02.871274 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d435034-4594-4d96-8c4b-bd9ac6b61082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:02.871544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d435034-4594-4d96-8c4b-bd9ac6b61082 admin admin] PATH_INFO: `/v3/users/e8a7a076be024c268556730b7b710579` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:02.872953 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d435034-4594-4d96-8c4b-bd9ac6b61082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8a7a076be024c268556730b7b710579)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:02.874913 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d435034-4594-4d96-8c4b-bd9ac6b61082 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:02.925400 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1049/2133] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:47:02 2022] POST /identity/v3/users => generated 292 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:02.930211 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef4de7-1b7d-4565-b3bb-b22823b07834 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:02.932073 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-1bef4de7-1b7d-4565-b3bb-b22823b07834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:02.944072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef4de7-1b7d-4565-b3bb-b22823b07834 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:02.949771 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ed152-4b6e-4f63-a9e2-42a9ae0ad95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:02.950566 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ed152-4b6e-4f63-a9e2-42a9ae0ad95a admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:02.950930 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ed152-4b6e-4f63-a9e2-42a9ae0ad95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:02.951286 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ed152-4b6e-4f63-a9e2-42a9ae0ad95a admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:02.952103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ed152-4b6e-4f63-a9e2-42a9ae0ad95a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:02.953779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ed152-4b6e-4f63-a9e2-42a9ae0ad95a admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:02.965698 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1050/2134] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:47:02.969591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ed152-4b6e-4f63-a9e2-42a9ae0ad95a admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:02.980009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ed152-4b6e-4f63-a9e2-42a9ae0ad95a admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:02.985701 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dab59e-5a50-43ec-b426-4e5fd7262258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:02.986985 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dab59e-5a50-43ec-b426-4e5fd7262258 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:02.987284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dab59e-5a50-43ec-b426-4e5fd7262258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:02.987562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dab59e-5a50-43ec-b426-4e5fd7262258 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:02.988290 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dab59e-5a50-43ec-b426-4e5fd7262258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:02.990105 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dab59e-5a50-43ec-b426-4e5fd7262258 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:03.001917 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1051/2135] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:03.005845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dab59e-5a50-43ec-b426-4e5fd7262258 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:03.016771 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dab59e-5a50-43ec-b426-4e5fd7262258 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:03.021632 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a85ce-5e10-43a7-ac39-55888c0ae2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:03.022515 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a85ce-5e10-43a7-ac39-55888c0ae2f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:03.022878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a85ce-5e10-43a7-ac39-55888c0ae2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:03.023158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a85ce-5e10-43a7-ac39-55888c0ae2f3 admin admin] PATH_INFO: `/v3/projects/20cb3d3e51444aeabe9ffa1bc9bef628/users/c2ad0b1f4b384397879f8865cad3cab2/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:03.039473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a85ce-5e10-43a7-ac39-55888c0ae2f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20cb3d3e51444aeabe9ffa1bc9bef628, user_id=c2ad0b1f4b384397879f8865cad3cab2, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:03.042579 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a85ce-5e10-43a7-ac39-55888c0ae2f3 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:03.099525 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3d435034-4594-4d96-8c4b-bd9ac6b61082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8a7a076be024c268556730b7b710579'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:03.116041 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3d435034-4594-4d96-8c4b-bd9ac6b61082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8a7a076be024c268556730b7b710579'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:03.117602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3d435034-4594-4d96-8c4b-bd9ac6b61082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8a7a076be024c268556730b7b710579'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:03.118334 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3d435034-4594-4d96-8c4b-bd9ac6b61082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8a7a076be024c268556730b7b710579'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:03.125161 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1052/2136] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:03 2022] PUT /identity/v3/projects/20cb3d3e51444aeabe9ffa1bc9bef628/users/c2ad0b1f4b384397879f8865cad3cab2/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:03.137036 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3d435034-4594-4d96-8c4b-bd9ac6b61082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8a7a076be024c268556730b7b710579'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:03.137972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debc1a3-78c8-42b6-a8dd-bf53f20b54e9 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:03.138274 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debc1a3-78c8-42b6-a8dd-bf53f20b54e9 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:03.139593 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debc1a3-78c8-42b6-a8dd-bf53f20b54e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:03.169839 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-2debc1a3-78c8-42b6-a8dd-bf53f20b54e9 None None] MFA Rules not processed for user `c2ad0b1f4b384397879f8865cad3cab2`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:03.171163 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-2debc1a3-78c8-42b6-a8dd-bf53f20b54e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:03.234397 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1085/2137] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:02 2022] DELETE /identity/v3/users/e8a7a076be024c268556730b7b710579 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:03.247463 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1053/2138] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:03.252046 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d435034-4594-4d96-8c4b-bd9ac6b61082 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:03.253825 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3d435034-4594-4d96-8c4b-bd9ac6b61082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:03.308099 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:03.342783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d435034-4594-4d96-8c4b-bd9ac6b61082 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:03.347581 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22208e9d-252b-4b76-bd73-d6c83e2f69fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:03.348515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22208e9d-252b-4b76-bd73-d6c83e2f69fc admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:03.348837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22208e9d-252b-4b76-bd73-d6c83e2f69fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:03.349118 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22208e9d-252b-4b76-bd73-d6c83e2f69fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:03.349827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22208e9d-252b-4b76-bd73-d6c83e2f69fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:03.352320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22208e9d-252b-4b76-bd73-d6c83e2f69fc admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:03.441914 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1086/2139] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:03 2022] POST /identity/v3/projects => generated 383 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:03.447127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2debc1a3-78c8-42b6-a8dd-bf53f20b54e9 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:03.533097 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2debc1a3-78c8-42b6-a8dd-bf53f20b54e9 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:03.608873 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4af99-1e7b-49ab-8fed-3c8f5b9806c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:03.609548 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4af99-1e7b-49ab-8fed-3c8f5b9806c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:03.609834 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4af99-1e7b-49ab-8fed-3c8f5b9806c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:03.610114 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4af99-1e7b-49ab-8fed-3c8f5b9806c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:03.610805 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4af99-1e7b-49ab-8fed-3c8f5b9806c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:03.612846 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4af99-1e7b-49ab-8fed-3c8f5b9806c9 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:03.622549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:03.680938 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22208e9d-252b-4b76-bd73-d6c83e2f69fc admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:03.683324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-22208e9d-252b-4b76-bd73-d6c83e2f69fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:03.707611 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22208e9d-252b-4b76-bd73-d6c83e2f69fc admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:03.712495 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1054/2140] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:47:03 2022] POST /identity/v3/users => generated 298 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:03.716661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4af99-1e7b-49ab-8fed-3c8f5b9806c9 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:03.717951 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5280b-04fa-43c7-b0ad-8d261664c8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:03.718852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5280b-04fa-43c7-b0ad-8d261664c8d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:03.719137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5280b-04fa-43c7-b0ad-8d261664c8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:03.719413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5280b-04fa-43c7-b0ad-8d261664c8d7 admin admin] PATH_INFO: `/v3/projects/353d4225356f4a3cb3d9177a92abf154` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:03.720656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5280b-04fa-43c7-b0ad-8d261664c8d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=353d4225356f4a3cb3d9177a92abf154)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:03.722574 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5280b-04fa-43c7-b0ad-8d261664c8d7 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:03.727590 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4af99-1e7b-49ab-8fed-3c8f5b9806c9 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:03.732705 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cafd8-e619-4142-9653-d86088b60d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:03.733440 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cafd8-e619-4142-9653-d86088b60d87 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:03.733791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cafd8-e619-4142-9653-d86088b60d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:03.734154 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cafd8-e619-4142-9653-d86088b60d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:03.734941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cafd8-e619-4142-9653-d86088b60d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:03.736693 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cafd8-e619-4142-9653-d86088b60d87 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:03.747634 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1055/2141] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:47:03.751438 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cafd8-e619-4142-9653-d86088b60d87 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:03.761011 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cafd8-e619-4142-9653-d86088b60d87 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:03.765658 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9efbb4e-7735-44f5-83e3-992a015b4b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:03.769205 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efbb4e-7735-44f5-83e3-992a015b4b58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:03.769205 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efbb4e-7735-44f5-83e3-992a015b4b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:03.769205 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efbb4e-7735-44f5-83e3-992a015b4b58 admin admin] PATH_INFO: `/v3/projects/6d11a3bbaa49426b961f1287a7ceb699/users/4097e8960fe64f96a41f4fe6671824f9/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:03.782887 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9efbb4e-7735-44f5-83e3-992a015b4b58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d11a3bbaa49426b961f1287a7ceb699, user_id=4097e8960fe64f96a41f4fe6671824f9, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:03.785574 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9efbb4e-7735-44f5-83e3-992a015b4b58 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:03.869869 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1056/2142] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:03 2022] PUT /identity/v3/projects/6d11a3bbaa49426b961f1287a7ceb699/users/4097e8960fe64f96a41f4fe6671824f9/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:03.873652 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9efbb4e-7735-44f5-83e3-992a015b4b58 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:03.878683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-bdd5280b-04fa-43c7-b0ad-8d261664c8d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '353d4225356f4a3cb3d9177a92abf154', 'request_id': 'req-bdd5280b-04fa-43c7-b0ad-8d261664c8d7'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:03.889507 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-bdd5280b-04fa-43c7-b0ad-8d261664c8d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '353d4225356f4a3cb3d9177a92abf154', 'request_id': 'req-bdd5280b-04fa-43c7-b0ad-8d261664c8d7'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:03.898505 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-bdd5280b-04fa-43c7-b0ad-8d261664c8d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '353d4225356f4a3cb3d9177a92abf154', 'request_id': 'req-bdd5280b-04fa-43c7-b0ad-8d261664c8d7'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:03.941363 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9efbb4e-7735-44f5-83e3-992a015b4b58 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:04.004634 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6414db1b-9ab9-4029-92f5-383ecdbad83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:04.004841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-bdd5280b-04fa-43c7-b0ad-8d261664c8d7 admin admin] The token cache is being invalidate because project 353d4225356f4a3cb3d9177a92abf154 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:04.005077 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-bdd5280b-04fa-43c7-b0ad-8d261664c8d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:04.005365 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414db1b-9ab9-4029-92f5-383ecdbad83c admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:04.005651 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414db1b-9ab9-4029-92f5-383ecdbad83c admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:04.005760 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-bdd5280b-04fa-43c7-b0ad-8d261664c8d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:04.005930 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414db1b-9ab9-4029-92f5-383ecdbad83c admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:04.006680 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6414db1b-9ab9-4029-92f5-383ecdbad83c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:04.007826 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1087/2143] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:03 2022] DELETE /identity/v3/projects/353d4225356f4a3cb3d9177a92abf154 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:04.008876 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6414db1b-9ab9-4029-92f5-383ecdbad83c admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:04.013648 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6dc970-b42d-40f4-ae29-69a0c30e8603 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:04.013942 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6dc970-b42d-40f4-ae29-69a0c30e8603 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:04.014230 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6dc970-b42d-40f4-ae29-69a0c30e8603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:04.019020 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1057/2144] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:47:04.023861 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6414db1b-9ab9-4029-92f5-383ecdbad83c admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:04.025710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-6414db1b-9ab9-4029-92f5-383ecdbad83c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:04.033303 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-da6dc970-b42d-40f4-ae29-69a0c30e8603 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:04.034571 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-da6dc970-b42d-40f4-ae29-69a0c30e8603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:04.038565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6414db1b-9ab9-4029-92f5-383ecdbad83c admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:04.043010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1088/2145] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:47:04.054908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d96591-aec2-4bad-85ee-d8c2f0ade1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:04.054908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d96591-aec2-4bad-85ee-d8c2f0ade1d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:04.054908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d96591-aec2-4bad-85ee-d8c2f0ade1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:04.054908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d96591-aec2-4bad-85ee-d8c2f0ade1d5 admin admin] PATH_INFO: `/v3/projects/6d11a3bbaa49426b961f1287a7ceb699/users/4097e8960fe64f96a41f4fe6671824f9/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:04.055233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d96591-aec2-4bad-85ee-d8c2f0ade1d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d11a3bbaa49426b961f1287a7ceb699, user_id=4097e8960fe64f96a41f4fe6671824f9, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:04.055233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d96591-aec2-4bad-85ee-d8c2f0ade1d5 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:04.055386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf3284-9b87-4488-828a-d6eef7543fb9 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:04.055386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf3284-9b87-4488-828a-d6eef7543fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:04.055386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf3284-9b87-4488-828a-d6eef7543fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:04.060109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1058/2146] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:04 2022] PUT /identity/v3/projects/6d11a3bbaa49426b961f1287a7ceb699/users/4097e8960fe64f96a41f4fe6671824f9/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:04.070973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f43d7-2a79-4ab6-8d98-34a255796cc6 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:04.071215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-19bf3284-9b87-4488-828a-d6eef7543fb9 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:04.071326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f43d7-2a79-4ab6-8d98-34a255796cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:04.071536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f43d7-2a79-4ab6-8d98-34a255796cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:04.072408 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-19bf3284-9b87-4488-828a-d6eef7543fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:04.098232 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-5b7f43d7-2a79-4ab6-8d98-34a255796cc6 None None] MFA Rules not processed for user `4097e8960fe64f96a41f4fe6671824f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:04.099434 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-5b7f43d7-2a79-4ab6-8d98-34a255796cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:04.136468 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1089/2147] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:04.140601 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf3284-9b87-4488-828a-d6eef7543fb9 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:04.150077 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf3284-9b87-4488-828a-d6eef7543fb9 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:04.157382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32acdbfc-fe00-4e29-9fca-73d051fe0f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:04.157967 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acdbfc-fe00-4e29-9fca-73d051fe0f60 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:04.158253 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acdbfc-fe00-4e29-9fca-73d051fe0f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:04.158531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acdbfc-fe00-4e29-9fca-73d051fe0f60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:04.161146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32acdbfc-fe00-4e29-9fca-73d051fe0f60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:47:04.161146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32acdbfc-fe00-4e29-9fca-73d051fe0f60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:04.161513 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32acdbfc-fe00-4e29-9fca-73d051fe0f60 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:04.174012 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1090/2148] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:47:04.178292 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32acdbfc-fe00-4e29-9fca-73d051fe0f60 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:04.178450 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1059/2149] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:47:04.182625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:04.191442 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f43d7-2a79-4ab6-8d98-34a255796cc6 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:04.193517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-5b7f43d7-2a79-4ab6-8d98-34a255796cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:04.193683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32acdbfc-fe00-4e29-9fca-73d051fe0f60 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:04.199402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1392ba7-fc72-4fc5-8979-169b9cc99a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:04.200057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1392ba7-fc72-4fc5-8979-169b9cc99a9c admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:04.200343 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1392ba7-fc72-4fc5-8979-169b9cc99a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:04.200653 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1392ba7-fc72-4fc5-8979-169b9cc99a9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:04.201343 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1392ba7-fc72-4fc5-8979-169b9cc99a9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:04.203386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1392ba7-fc72-4fc5-8979-169b9cc99a9c admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:04.208433 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:04.260228 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f43d7-2a79-4ab6-8d98-34a255796cc6 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:04.263828 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303c1ae-c554-4603-bf88-9271373f4e9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:04.264478 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303c1ae-c554-4603-bf88-9271373f4e9e service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:04.264787 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303c1ae-c554-4603-bf88-9271373f4e9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:04.265091 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303c1ae-c554-4603-bf88-9271373f4e9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:04.274479 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303c1ae-c554-4603-bf88-9271373f4e9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:04.275912 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303c1ae-c554-4603-bf88-9271373f4e9e service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:04.289576 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1060/2150] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:47:04.290457 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1091/2151] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:04 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:47:04.294723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303c1ae-c554-4603-bf88-9271373f4e9e service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:04.306089 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303c1ae-c554-4603-bf88-9271373f4e9e service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:04.367256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab2a1d-552b-4cbc-a30a-f39bced20ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:04.372107 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab2a1d-552b-4cbc-a30a-f39bced20ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:04.372107 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab2a1d-552b-4cbc-a30a-f39bced20ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:04.372107 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab2a1d-552b-4cbc-a30a-f39bced20ff0 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:04.372107 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab2a1d-552b-4cbc-a30a-f39bced20ff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:04.372107 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab2a1d-552b-4cbc-a30a-f39bced20ff0 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:04.387458 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:04.389692 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1392ba7-fc72-4fc5-8979-169b9cc99a9c admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:04.448201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1392ba7-fc72-4fc5-8979-169b9cc99a9c admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:04.453660 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1061/2152] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:47:04 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:47:04.454688 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab2a1d-552b-4cbc-a30a-f39bced20ff0 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:04.455145 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de72372-7f82-4b08-9346-1bc798d7168a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:04.455821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de72372-7f82-4b08-9346-1bc798d7168a service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:04.456110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de72372-7f82-4b08-9346-1bc798d7168a service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:04.456387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de72372-7f82-4b08-9346-1bc798d7168a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:04.458215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-4de72372-7f82-4b08-9346-1bc798d7168a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:04.465000 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab2a1d-552b-4cbc-a30a-f39bced20ff0 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:04.470142 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f24ef9-8626-4006-ba5b-bb9787e45792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:04.475556 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f24ef9-8626-4006-ba5b-bb9787e45792 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:04.475556 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f24ef9-8626-4006-ba5b-bb9787e45792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:04.475556 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f24ef9-8626-4006-ba5b-bb9787e45792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:04.475556 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f24ef9-8626-4006-ba5b-bb9787e45792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:04.475556 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f24ef9-8626-4006-ba5b-bb9787e45792 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:04.484536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1062/2153] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:47:04.488568 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f24ef9-8626-4006-ba5b-bb9787e45792 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:04.500034 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f24ef9-8626-4006-ba5b-bb9787e45792 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:04.503364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7471e403-ba0a-4997-84f8-c5f3516e3eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:04.504022 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471e403-ba0a-4997-84f8-c5f3516e3eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:04.504308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471e403-ba0a-4997-84f8-c5f3516e3eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:04.504703 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471e403-ba0a-4997-84f8-c5f3516e3eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:04.505321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7471e403-ba0a-4997-84f8-c5f3516e3eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:04.506924 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7471e403-ba0a-4997-84f8-c5f3516e3eb7 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:04.517540 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1063/2154] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:47:04.521602 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7471e403-ba0a-4997-84f8-c5f3516e3eb7 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:04.524545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de72372-7f82-4b08-9346-1bc798d7168a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:04.526023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de72372-7f82-4b08-9346-1bc798d7168a service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:04.535846 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1092/2155] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:04.542254 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7471e403-ba0a-4997-84f8-c5f3516e3eb7 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:04.547513 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713bd74-c37e-457e-b216-cb32baa9647c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:04.548367 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713bd74-c37e-457e-b216-cb32baa9647c admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:04.549093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713bd74-c37e-457e-b216-cb32baa9647c admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:04.551204 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713bd74-c37e-457e-b216-cb32baa9647c admin admin] PATH_INFO: `/v3/projects/c7092bb1c14746b3ba49c0492b141c47/users/316ec88e7720400987d254a91f3bb490/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:04.566250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713bd74-c37e-457e-b216-cb32baa9647c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7092bb1c14746b3ba49c0492b141c47, user_id=316ec88e7720400987d254a91f3bb490, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:04.569009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713bd74-c37e-457e-b216-cb32baa9647c admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:04.576860 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:04.640548 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1064/2156] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:04 2022] PUT /identity/v3/projects/c7092bb1c14746b3ba49c0492b141c47/users/316ec88e7720400987d254a91f3bb490/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:04.654050 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6fae9-c10a-4465-8d61-cb22be4943e1 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:04.654566 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6fae9-c10a-4465-8d61-cb22be4943e1 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:04.654932 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6fae9-c10a-4465-8d61-cb22be4943e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:04.691822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-d9a6fae9-c10a-4465-8d61-cb22be4943e1 None None] MFA Rules not processed for user `316ec88e7720400987d254a91f3bb490`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:04.693446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d9a6fae9-c10a-4465-8d61-cb22be4943e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:04.771732 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1093/2157] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:04.777492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713bd74-c37e-457e-b216-cb32baa9647c admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:04.798157 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:04.822819 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:04.844927 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713bd74-c37e-457e-b216-cb32baa9647c admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:04.849915 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38369678-491f-418b-829f-e3819dd0533c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:04.850573 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38369678-491f-418b-829f-e3819dd0533c admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:04.851588 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38369678-491f-418b-829f-e3819dd0533c admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:04.851835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38369678-491f-418b-829f-e3819dd0533c admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:04.852537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38369678-491f-418b-829f-e3819dd0533c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:04.854591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38369678-491f-418b-829f-e3819dd0533c admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:04.927163 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1065/2158] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:04 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:47:04.931541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6fae9-c10a-4465-8d61-cb22be4943e1 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:04.942338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6fae9-c10a-4465-8d61-cb22be4943e1 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:05.004061 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863eb16-3e29-44b7-a69d-ca78fde58b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:05.004780 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863eb16-3e29-44b7-a69d-ca78fde58b73 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:05.005068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863eb16-3e29-44b7-a69d-ca78fde58b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:05.005740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863eb16-3e29-44b7-a69d-ca78fde58b73 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:05.006214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863eb16-3e29-44b7-a69d-ca78fde58b73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:05.008000 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863eb16-3e29-44b7-a69d-ca78fde58b73 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:05.079778 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1094/2159] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:47:04 2022] POST /identity/v3/users => generated 296 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:47:05.083955 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38369678-491f-418b-829f-e3819dd0533c admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:05.095786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38369678-491f-418b-829f-e3819dd0533c admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:05.100544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6ec04-836a-4603-a02e-b7d57fa0df12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:05.102224 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6ec04-836a-4603-a02e-b7d57fa0df12 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:05.102508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6ec04-836a-4603-a02e-b7d57fa0df12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:05.102877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6ec04-836a-4603-a02e-b7d57fa0df12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:05.103659 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6ec04-836a-4603-a02e-b7d57fa0df12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:05.105937 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6ec04-836a-4603-a02e-b7d57fa0df12 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:05.122329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1066/2160] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:47:05.129670 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863eb16-3e29-44b7-a69d-ca78fde58b73 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:05.140099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863eb16-3e29-44b7-a69d-ca78fde58b73 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:05.144845 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a6afe-4eb6-471f-aeb3-2dedf7a886c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:05.145763 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a6afe-4eb6-471f-aeb3-2dedf7a886c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:05.146046 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a6afe-4eb6-471f-aeb3-2dedf7a886c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:05.146353 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a6afe-4eb6-471f-aeb3-2dedf7a886c4 admin admin] PATH_INFO: `/v3/projects/afd9fe40db6e4241a5e30d75ba3d7137/users/e77590d2a9f44d86a1cff13167c050eb/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:05.162882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a6afe-4eb6-471f-aeb3-2dedf7a886c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afd9fe40db6e4241a5e30d75ba3d7137, user_id=e77590d2a9f44d86a1cff13167c050eb, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:05.166296 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a6afe-4eb6-471f-aeb3-2dedf7a886c4 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:05.170381 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:05.265576 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1095/2161] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:05 2022] PUT /identity/v3/projects/afd9fe40db6e4241a5e30d75ba3d7137/users/e77590d2a9f44d86a1cff13167c050eb/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:05.271015 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6ec04-836a-4603-a02e-b7d57fa0df12 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:05.292225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6ec04-836a-4603-a02e-b7d57fa0df12 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:05.360188 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:05.370952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876654f-8c3b-4d28-ace6-89c92a910598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:05.371580 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876654f-8c3b-4d28-ace6-89c92a910598 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:05.371865 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876654f-8c3b-4d28-ace6-89c92a910598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:05.372154 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876654f-8c3b-4d28-ace6-89c92a910598 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:05.372922 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876654f-8c3b-4d28-ace6-89c92a910598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:05.374626 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876654f-8c3b-4d28-ace6-89c92a910598 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:05.387110 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1067/2162] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:05.391390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a6afe-4eb6-471f-aeb3-2dedf7a886c4 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:05.404343 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a6afe-4eb6-471f-aeb3-2dedf7a886c4 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:05.409618 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7888c96-146c-4903-af82-79bc88636d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:05.410920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7888c96-146c-4903-af82-79bc88636d58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:05.411531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7888c96-146c-4903-af82-79bc88636d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:05.412438 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7888c96-146c-4903-af82-79bc88636d58 admin admin] PATH_INFO: `/v3/projects/afd9fe40db6e4241a5e30d75ba3d7137/users/e77590d2a9f44d86a1cff13167c050eb/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:05.416945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7888c96-146c-4903-af82-79bc88636d58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afd9fe40db6e4241a5e30d75ba3d7137, user_id=e77590d2a9f44d86a1cff13167c050eb, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:05.419614 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7888c96-146c-4903-af82-79bc88636d58 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:05.430505 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1096/2163] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:05 2022] PUT /identity/v3/projects/afd9fe40db6e4241a5e30d75ba3d7137/users/e77590d2a9f44d86a1cff13167c050eb/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:05.440743 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd80990-d256-4fd3-81c8-07f9674b91c4 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:05.441062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd80990-d256-4fd3-81c8-07f9674b91c4 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:05.441341 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd80990-d256-4fd3-81c8-07f9674b91c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:05.469318 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-2fd80990-d256-4fd3-81c8-07f9674b91c4 None None] MFA Rules not processed for user `e77590d2a9f44d86a1cff13167c050eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:05.470523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-2fd80990-d256-4fd3-81c8-07f9674b91c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:05.502418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:05.541539 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1068/2164] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:05.552733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7888c96-146c-4903-af82-79bc88636d58 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:05.554587 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d7888c96-146c-4903-af82-79bc88636d58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:05.615826 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7888c96-146c-4903-af82-79bc88636d58 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:05.619922 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a846c-02de-4aa5-bdcf-d911f3433d8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:05.620682 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a846c-02de-4aa5-bdcf-d911f3433d8a service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:05.620967 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a846c-02de-4aa5-bdcf-d911f3433d8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:05.621245 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a846c-02de-4aa5-bdcf-d911f3433d8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:05.630831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a846c-02de-4aa5-bdcf-d911f3433d8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:05.632598 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a846c-02de-4aa5-bdcf-d911f3433d8a service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:05.642299 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1097/2165] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:06.157442 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd80990-d256-4fd3-81c8-07f9674b91c4 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:06.172425 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:06.222477 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd80990-d256-4fd3-81c8-07f9674b91c4 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:06.227686 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44ca41-2b5e-4c24-bfdf-645de0c2c7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:06.228590 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44ca41-2b5e-4c24-bfdf-645de0c2c7b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:06.228894 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44ca41-2b5e-4c24-bfdf-645de0c2c7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:06.229225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44ca41-2b5e-4c24-bfdf-645de0c2c7b1 admin admin] PATH_INFO: `/v3/users/316ec88e7720400987d254a91f3bb490` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:06.230475 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44ca41-2b5e-4c24-bfdf-645de0c2c7b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=316ec88e7720400987d254a91f3bb490)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:06.232553 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44ca41-2b5e-4c24-bfdf-645de0c2c7b1 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:06.399811 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-8e44ca41-2b5e-4c24-bfdf-645de0c2c7b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '316ec88e7720400987d254a91f3bb490'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:06.412025 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-8e44ca41-2b5e-4c24-bfdf-645de0c2c7b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '316ec88e7720400987d254a91f3bb490'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:06.456097 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-8e44ca41-2b5e-4c24-bfdf-645de0c2c7b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '316ec88e7720400987d254a91f3bb490'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:06.456879 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-8e44ca41-2b5e-4c24-bfdf-645de0c2c7b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '316ec88e7720400987d254a91f3bb490'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:06.457507 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-8e44ca41-2b5e-4c24-bfdf-645de0c2c7b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '316ec88e7720400987d254a91f3bb490'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:06.472118 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1069/2166] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:06 2022] DELETE /identity/v3/users/316ec88e7720400987d254a91f3bb490 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:06.477258 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700fd87-7574-42dd-ac83-916f52e71f30 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:06.478815 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700fd87-7574-42dd-ac83-916f52e71f30 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:06.479539 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700fd87-7574-42dd-ac83-916f52e71f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:06.501485 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-4700fd87-7574-42dd-ac83-916f52e71f30 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:06.502715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-4700fd87-7574-42dd-ac83-916f52e71f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:06.551006 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:06.563815 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1098/2167] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:06.573561 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44ca41-2b5e-4c24-bfdf-645de0c2c7b1 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:06.575079 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-8e44ca41-2b5e-4c24-bfdf-645de0c2c7b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:06.635714 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44ca41-2b5e-4c24-bfdf-645de0c2c7b1 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:06.639521 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8747c-3b9c-4691-a8a2-4123160c337e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:06.640223 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8747c-3b9c-4691-a8a2-4123160c337e service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:06.640499 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8747c-3b9c-4691-a8a2-4123160c337e service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:06.640820 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8747c-3b9c-4691-a8a2-4123160c337e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:06.652134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8747c-3b9c-4691-a8a2-4123160c337e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:06.653696 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8747c-3b9c-4691-a8a2-4123160c337e service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:06.664478 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1070/2168] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:07.131237 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700fd87-7574-42dd-ac83-916f52e71f30 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:07.132806 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-4700fd87-7574-42dd-ac83-916f52e71f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:07.150730 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700fd87-7574-42dd-ac83-916f52e71f30 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:07.156534 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e563f6-131d-4463-8890-fe818bc0f7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:07.163068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e563f6-131d-4463-8890-fe818bc0f7a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:07.163068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e563f6-131d-4463-8890-fe818bc0f7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:07.163068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e563f6-131d-4463-8890-fe818bc0f7a2 admin admin] PATH_INFO: `/v3/projects/c7092bb1c14746b3ba49c0492b141c47` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:07.163068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e563f6-131d-4463-8890-fe818bc0f7a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7092bb1c14746b3ba49c0492b141c47)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:07.163068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e563f6-131d-4463-8890-fe818bc0f7a2 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:07.343185 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e1e563f6-131d-4463-8890-fe818bc0f7a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7092bb1c14746b3ba49c0492b141c47', 'request_id': 'req-e1e563f6-131d-4463-8890-fe818bc0f7a2'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:07.355678 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e1e563f6-131d-4463-8890-fe818bc0f7a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7092bb1c14746b3ba49c0492b141c47', 'request_id': 'req-e1e563f6-131d-4463-8890-fe818bc0f7a2'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:07.355678 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e1e563f6-131d-4463-8890-fe818bc0f7a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7092bb1c14746b3ba49c0492b141c47', 'request_id': 'req-e1e563f6-131d-4463-8890-fe818bc0f7a2'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:07.428505 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e1e563f6-131d-4463-8890-fe818bc0f7a2 admin admin] The token cache is being invalidate because project c7092bb1c14746b3ba49c0492b141c47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:07.429687 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e1e563f6-131d-4463-8890-fe818bc0f7a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:07.431405 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e1e563f6-131d-4463-8890-fe818bc0f7a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:07.433415 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1099/2169] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:07 2022] DELETE /identity/v3/projects/c7092bb1c14746b3ba49c0492b141c47 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:07.439211 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8747c-3b9c-4691-a8a2-4123160c337e service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:07.458617 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8747c-3b9c-4691-a8a2-4123160c337e service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:07.497377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:07.536620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f0351-3a68-4be9-83f0-a9fcf358b683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:07.537559 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f0351-3a68-4be9-83f0-a9fcf358b683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:07.537847 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f0351-3a68-4be9-83f0-a9fcf358b683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:07.538158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f0351-3a68-4be9-83f0-a9fcf358b683 admin admin] PATH_INFO: `/v3/users/e77590d2a9f44d86a1cff13167c050eb` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:07.539803 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1f0351-3a68-4be9-83f0-a9fcf358b683 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e77590d2a9f44d86a1cff13167c050eb)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:07.542020 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1f0351-3a68-4be9-83f0-a9fcf358b683 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:07.699007 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9a1f0351-3a68-4be9-83f0-a9fcf358b683 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e77590d2a9f44d86a1cff13167c050eb'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:07.712599 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9a1f0351-3a68-4be9-83f0-a9fcf358b683 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e77590d2a9f44d86a1cff13167c050eb'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:07.775213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e563f6-131d-4463-8890-fe818bc0f7a2 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:07.776185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9a1f0351-3a68-4be9-83f0-a9fcf358b683 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e77590d2a9f44d86a1cff13167c050eb'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:07.776695 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e1e563f6-131d-4463-8890-fe818bc0f7a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:07.776851 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9a1f0351-3a68-4be9-83f0-a9fcf358b683 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e77590d2a9f44d86a1cff13167c050eb'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:07.777479 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9a1f0351-3a68-4be9-83f0-a9fcf358b683 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e77590d2a9f44d86a1cff13167c050eb'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:07.789880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e563f6-131d-4463-8890-fe818bc0f7a2 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:07.791504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1071/2170] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:07 2022] DELETE /identity/v3/users/e77590d2a9f44d86a1cff13167c050eb => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:07.860040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b858c-6253-45fd-ab60-4c4db46b79dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:07.861069 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b858c-6253-45fd-ab60-4c4db46b79dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:07.861359 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b858c-6253-45fd-ab60-4c4db46b79dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:07.861640 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b858c-6253-45fd-ab60-4c4db46b79dc admin admin] PATH_INFO: `/v3/users/c2ad0b1f4b384397879f8865cad3cab2` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:07.862873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b858c-6253-45fd-ab60-4c4db46b79dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2ad0b1f4b384397879f8865cad3cab2)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:07.865076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b858c-6253-45fd-ab60-4c4db46b79dc admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:08.099354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6a8b858c-6253-45fd-ab60-4c4db46b79dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2ad0b1f4b384397879f8865cad3cab2'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:08.112545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6a8b858c-6253-45fd-ab60-4c4db46b79dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2ad0b1f4b384397879f8865cad3cab2'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:08.113366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6a8b858c-6253-45fd-ab60-4c4db46b79dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2ad0b1f4b384397879f8865cad3cab2'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:08.114435 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6a8b858c-6253-45fd-ab60-4c4db46b79dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2ad0b1f4b384397879f8865cad3cab2'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:08.126578 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6a8b858c-6253-45fd-ab60-4c4db46b79dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2ad0b1f4b384397879f8865cad3cab2'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:08.140548 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:08.235224 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f0351-3a68-4be9-83f0-a9fcf358b683 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:08.237014 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-9a1f0351-3a68-4be9-83f0-a9fcf358b683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:08.252387 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f0351-3a68-4be9-83f0-a9fcf358b683 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:08.263359 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1100/2171] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:07 2022] DELETE /identity/v3/users/c2ad0b1f4b384397879f8865cad3cab2 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:08.266879 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca00bab-acfc-4026-9b46-811a060e79bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:08.267814 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff60473-a111-4d81-bb5b-d53d29cdb58d None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:08.268053 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca00bab-acfc-4026-9b46-811a060e79bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:08.268169 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff60473-a111-4d81-bb5b-d53d29cdb58d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:08.268330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca00bab-acfc-4026-9b46-811a060e79bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:08.268442 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff60473-a111-4d81-bb5b-d53d29cdb58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:08.268639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca00bab-acfc-4026-9b46-811a060e79bd admin admin] PATH_INFO: `/v3/projects/afd9fe40db6e4241a5e30d75ba3d7137` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:08.274041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca00bab-acfc-4026-9b46-811a060e79bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=afd9fe40db6e4241a5e30d75ba3d7137)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:08.276053 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca00bab-acfc-4026-9b46-811a060e79bd admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:08.287518 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-aff60473-a111-4d81-bb5b-d53d29cdb58d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:08.289665 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-aff60473-a111-4d81-bb5b-d53d29cdb58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:08.360451 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1101/2172] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:08.369740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff60473-a111-4d81-bb5b-d53d29cdb58d None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:08.371129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-aff60473-a111-4d81-bb5b-d53d29cdb58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:08.432541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff60473-a111-4d81-bb5b-d53d29cdb58d None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:08.457614 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-eca00bab-acfc-4026-9b46-811a060e79bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afd9fe40db6e4241a5e30d75ba3d7137', 'request_id': 'req-eca00bab-acfc-4026-9b46-811a060e79bd'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:08.490772 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84b9f5-b8be-4a56-bdb9-b6ff1181ed25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:08.492008 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84b9f5-b8be-4a56-bdb9-b6ff1181ed25 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:08.492297 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84b9f5-b8be-4a56-bdb9-b6ff1181ed25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:08.492609 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84b9f5-b8be-4a56-bdb9-b6ff1181ed25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:08.518747 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-eca00bab-acfc-4026-9b46-811a060e79bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afd9fe40db6e4241a5e30d75ba3d7137', 'request_id': 'req-eca00bab-acfc-4026-9b46-811a060e79bd'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:08.525715 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-eca00bab-acfc-4026-9b46-811a060e79bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afd9fe40db6e4241a5e30d75ba3d7137', 'request_id': 'req-eca00bab-acfc-4026-9b46-811a060e79bd'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:08.536936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-eca00bab-acfc-4026-9b46-811a060e79bd admin admin] The token cache is being invalidate because project afd9fe40db6e4241a5e30d75ba3d7137 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:08.537228 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-eca00bab-acfc-4026-9b46-811a060e79bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:08.537892 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-eca00bab-acfc-4026-9b46-811a060e79bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:08.539537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1072/2173] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:08 2022] DELETE /identity/v3/projects/afd9fe40db6e4241a5e30d75ba3d7137 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:08.543248 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677db66-ff24-4881-95bf-2421f2916993 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:08.543248 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677db66-ff24-4881-95bf-2421f2916993 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:08.543248 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677db66-ff24-4881-95bf-2421f2916993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:08.563343 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-9677db66-ff24-4881-95bf-2421f2916993 None None] MFA Rules not processed for user `84a5fb262284400591cf29dd13771ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:08.563928 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84b9f5-b8be-4a56-bdb9-b6ff1181ed25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:08.564751 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-9677db66-ff24-4881-95bf-2421f2916993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:08.567356 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84b9f5-b8be-4a56-bdb9-b6ff1181ed25 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:08.579773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1102/2174] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:47:08.581953 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5d8cb-382c-4c68-b3c5-e383ca2381d7 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:08.582551 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5d8cb-382c-4c68-b3c5-e383ca2381d7 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:08.583220 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5d8cb-382c-4c68-b3c5-e383ca2381d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:08.601455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:08.604229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-00a5d8cb-382c-4c68-b3c5-e383ca2381d7 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:08.605836 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-00a5d8cb-382c-4c68-b3c5-e383ca2381d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:08.616738 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1103/2175] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:08.624950 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3753c-f8f3-4b5a-abd3-affef9b142f2 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:08.625413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3753c-f8f3-4b5a-abd3-affef9b142f2 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:08.625684 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3753c-f8f3-4b5a-abd3-affef9b142f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:08.626973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1073/2176] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:08.650898 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-0fb3753c-f8f3-4b5a-abd3-affef9b142f2 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:08.652195 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-0fb3753c-f8f3-4b5a-abd3-affef9b142f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:08.664628 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1104/2177] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:08.668540 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677db66-ff24-4881-95bf-2421f2916993 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:08.681417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677db66-ff24-4881-95bf-2421f2916993 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:08.694317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8fe6a-d72a-4734-9882-146893b12400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:08.695688 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8fe6a-d72a-4734-9882-146893b12400 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:08.695982 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8fe6a-d72a-4734-9882-146893b12400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:08.696268 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8fe6a-d72a-4734-9882-146893b12400 admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:08.697059 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8fe6a-d72a-4734-9882-146893b12400 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:47:08.697371 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8fe6a-d72a-4734-9882-146893b12400 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:08.698983 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8fe6a-d72a-4734-9882-146893b12400 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:08.706203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:08.712661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1074/2178] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:47:08.716754 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3753c-f8f3-4b5a-abd3-affef9b142f2 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:08.727023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3753c-f8f3-4b5a-abd3-affef9b142f2 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:08.733089 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a7fc3-8b73-4aaa-a297-5e835797efc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:08.733704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a7fc3-8b73-4aaa-a297-5e835797efc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:08.734491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a7fc3-8b73-4aaa-a297-5e835797efc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:08.734491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a7fc3-8b73-4aaa-a297-5e835797efc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:08.735109 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a7fc3-8b73-4aaa-a297-5e835797efc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:08.737216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a7fc3-8b73-4aaa-a297-5e835797efc6 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:08.840636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1105/2179] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:08 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:08.844844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8fe6a-d72a-4734-9882-146893b12400 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:08.855486 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8fe6a-d72a-4734-9882-146893b12400 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:08.921748 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d339611-7645-459b-8074-a224d2db9f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:08.923795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d339611-7645-459b-8074-a224d2db9f0c admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:08.923795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d339611-7645-459b-8074-a224d2db9f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:08.924122 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d339611-7645-459b-8074-a224d2db9f0c admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:08.930933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d339611-7645-459b-8074-a224d2db9f0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:08.934951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d339611-7645-459b-8074-a224d2db9f0c admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:09.030138 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1075/2180] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:47:08 2022] POST /identity/v3/users => generated 285 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:09.034212 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a7fc3-8b73-4aaa-a297-5e835797efc6 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:09.045744 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a7fc3-8b73-4aaa-a297-5e835797efc6 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:09.050837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbaa744-711d-4b97-b975-48023e1b1302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:09.051635 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbaa744-711d-4b97-b975-48023e1b1302 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:09.052065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbaa744-711d-4b97-b975-48023e1b1302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:09.059555 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbaa744-711d-4b97-b975-48023e1b1302 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:09.060608 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbaa744-711d-4b97-b975-48023e1b1302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:09.065727 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbaa744-711d-4b97-b975-48023e1b1302 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:09.081116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1106/2181] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:09.085684 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d339611-7645-459b-8074-a224d2db9f0c admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:09.096342 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d339611-7645-459b-8074-a224d2db9f0c admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:09.101732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b080a2-2872-44ce-aff7-7419100eb199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:09.102420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b080a2-2872-44ce-aff7-7419100eb199 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:09.102708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b080a2-2872-44ce-aff7-7419100eb199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:09.102986 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b080a2-2872-44ce-aff7-7419100eb199 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:09.103719 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b080a2-2872-44ce-aff7-7419100eb199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:09.105381 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b080a2-2872-44ce-aff7-7419100eb199 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:09.115597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1076/2182] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:09.120035 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbaa744-711d-4b97-b975-48023e1b1302 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:09.130634 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbaa744-711d-4b97-b975-48023e1b1302 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:09.135849 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2678c0f-ed96-4540-892a-c7fa8da296f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:09.136767 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2678c0f-ed96-4540-892a-c7fa8da296f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:09.137059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2678c0f-ed96-4540-892a-c7fa8da296f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:09.137337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2678c0f-ed96-4540-892a-c7fa8da296f9 admin admin] PATH_INFO: `/v3/projects/81bfe73155c0422fb29425bc4b3ae4ff/users/b2b4a938daa0464a9be11627edf0e6d9/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:09.153455 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2678c0f-ed96-4540-892a-c7fa8da296f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81bfe73155c0422fb29425bc4b3ae4ff, user_id=b2b4a938daa0464a9be11627edf0e6d9, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:09.159710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2678c0f-ed96-4540-892a-c7fa8da296f9 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:09.160603 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b080a2-2872-44ce-aff7-7419100eb199 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:09.162404 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-94b080a2-2872-44ce-aff7-7419100eb199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:09.182631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b080a2-2872-44ce-aff7-7419100eb199 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:09.189028 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb3f91-511d-40a1-bb3b-be0672294285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:09.190523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb3f91-511d-40a1-bb3b-be0672294285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:09.191004 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb3f91-511d-40a1-bb3b-be0672294285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:09.191333 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb3f91-511d-40a1-bb3b-be0672294285 admin admin] PATH_INFO: `/v3/projects/20cb3d3e51444aeabe9ffa1bc9bef628` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:09.198185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb3f91-511d-40a1-bb3b-be0672294285 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20cb3d3e51444aeabe9ffa1bc9bef628)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:09.200627 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb3f91-511d-40a1-bb3b-be0672294285 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:09.246451 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1107/2183] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:09 2022] PUT /identity/v3/projects/81bfe73155c0422fb29425bc4b3ae4ff/users/b2b4a938daa0464a9be11627edf0e6d9/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:09.257931 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd6e0b-2079-4701-a31d-597950bec8b1 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:09.258203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd6e0b-2079-4701-a31d-597950bec8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:09.258670 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd6e0b-2079-4701-a31d-597950bec8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:09.286237 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-cacd6e0b-2079-4701-a31d-597950bec8b1 None None] MFA Rules not processed for user `b2b4a938daa0464a9be11627edf0e6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:09.287445 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-cacd6e0b-2079-4701-a31d-597950bec8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:09.341448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9dfb3f91-511d-40a1-bb3b-be0672294285 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20cb3d3e51444aeabe9ffa1bc9bef628', 'request_id': 'req-9dfb3f91-511d-40a1-bb3b-be0672294285'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:09.393934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1108/2184] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:09 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:47:09.401762 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9dfb3f91-511d-40a1-bb3b-be0672294285 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20cb3d3e51444aeabe9ffa1bc9bef628', 'request_id': 'req-9dfb3f91-511d-40a1-bb3b-be0672294285'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:09.408459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd6e0b-2079-4701-a31d-597950bec8b1 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:09.410052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9dfb3f91-511d-40a1-bb3b-be0672294285 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20cb3d3e51444aeabe9ffa1bc9bef628', 'request_id': 'req-9dfb3f91-511d-40a1-bb3b-be0672294285'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:09.410340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-cacd6e0b-2079-4701-a31d-597950bec8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:09.424552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9dfb3f91-511d-40a1-bb3b-be0672294285 admin admin] The token cache is being invalidate because project 20cb3d3e51444aeabe9ffa1bc9bef628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:09.425047 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9dfb3f91-511d-40a1-bb3b-be0672294285 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:09.425925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9dfb3f91-511d-40a1-bb3b-be0672294285 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:09.427647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1077/2185] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:09 2022] DELETE /identity/v3/projects/20cb3d3e51444aeabe9ffa1bc9bef628 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:09.431967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb3f91-511d-40a1-bb3b-be0672294285 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:09.433405 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-9dfb3f91-511d-40a1-bb3b-be0672294285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:09.484766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd6e0b-2079-4701-a31d-597950bec8b1 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:09.532661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb3f91-511d-40a1-bb3b-be0672294285 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:09.533390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:09.540417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fcfd5d-fb98-4d46-9437-45f1b4e3c054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:09.542150 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcfd5d-fb98-4d46-9437-45f1b4e3c054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:09.542591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcfd5d-fb98-4d46-9437-45f1b4e3c054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:09.543126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcfd5d-fb98-4d46-9437-45f1b4e3c054 admin admin] PATH_INFO: `/v3/users/4097e8960fe64f96a41f4fe6671824f9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:09.545412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fcfd5d-fb98-4d46-9437-45f1b4e3c054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4097e8960fe64f96a41f4fe6671824f9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:09.547882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fcfd5d-fb98-4d46-9437-45f1b4e3c054 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:09.552616 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a00a18-a516-47e1-a0fb-89f1213e4d7a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '983801bb18924fe4b3150693765eb13b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:09.553332 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a00a18-a516-47e1-a0fb-89f1213e4d7a service glance] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:09.554430 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a00a18-a516-47e1-a0fb-89f1213e4d7a service glance] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:09.554902 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a00a18-a516-47e1-a0fb-89f1213e4d7a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:09.557187 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-81a00a18-a516-47e1-a0fb-89f1213e4d7a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:09.605577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:09.619012 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a00a18-a516-47e1-a0fb-89f1213e4d7a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:09.621120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a00a18-a516-47e1-a0fb-89f1213e4d7a service glance] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:09.635003 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1109/2186] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:09 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:09.707731 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-b8fcfd5d-fb98-4d46-9437-45f1b4e3c054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4097e8960fe64f96a41f4fe6671824f9'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:09.720476 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-b8fcfd5d-fb98-4d46-9437-45f1b4e3c054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4097e8960fe64f96a41f4fe6671824f9'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:09.771839 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-b8fcfd5d-fb98-4d46-9437-45f1b4e3c054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4097e8960fe64f96a41f4fe6671824f9'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:09.773210 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-b8fcfd5d-fb98-4d46-9437-45f1b4e3c054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4097e8960fe64f96a41f4fe6671824f9'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:09.779133 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-b8fcfd5d-fb98-4d46-9437-45f1b4e3c054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4097e8960fe64f96a41f4fe6671824f9'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:09.796629 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1078/2187] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:09 2022] DELETE /identity/v3/users/4097e8960fe64f96a41f4fe6671824f9 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:10.436125 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a00a18-a516-47e1-a0fb-89f1213e4d7a service glance] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:10.449840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a00a18-a516-47e1-a0fb-89f1213e4d7a service glance] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:10.527022 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b50f0f-ed41-4bbc-848a-489ee8ab40c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:10.527932 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b50f0f-ed41-4bbc-848a-489ee8ab40c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:10.528220 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b50f0f-ed41-4bbc-848a-489ee8ab40c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:10.528636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b50f0f-ed41-4bbc-848a-489ee8ab40c4 admin admin] PATH_INFO: `/v3/projects/6d11a3bbaa49426b961f1287a7ceb699` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:10.529927 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b50f0f-ed41-4bbc-848a-489ee8ab40c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d11a3bbaa49426b961f1287a7ceb699)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:10.532054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b50f0f-ed41-4bbc-848a-489ee8ab40c4 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:10.699450 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c0b50f0f-ed41-4bbc-848a-489ee8ab40c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d11a3bbaa49426b961f1287a7ceb699', 'request_id': 'req-c0b50f0f-ed41-4bbc-848a-489ee8ab40c4'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:10.707557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c0b50f0f-ed41-4bbc-848a-489ee8ab40c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d11a3bbaa49426b961f1287a7ceb699', 'request_id': 'req-c0b50f0f-ed41-4bbc-848a-489ee8ab40c4'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:10.714337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c0b50f0f-ed41-4bbc-848a-489ee8ab40c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d11a3bbaa49426b961f1287a7ceb699', 'request_id': 'req-c0b50f0f-ed41-4bbc-848a-489ee8ab40c4'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:10.795965 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c0b50f0f-ed41-4bbc-848a-489ee8ab40c4 admin admin] The token cache is being invalidate because project 6d11a3bbaa49426b961f1287a7ceb699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:10.795965 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c0b50f0f-ed41-4bbc-848a-489ee8ab40c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:10.796637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c0b50f0f-ed41-4bbc-848a-489ee8ab40c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:10.799413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1110/2188] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:10 2022] DELETE /identity/v3/projects/6d11a3bbaa49426b961f1287a7ceb699 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:10.806331 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fd177-51ee-4fac-abb1-0d392f0d7cf7 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:10.806852 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fd177-51ee-4fac-abb1-0d392f0d7cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:10.806945 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fd177-51ee-4fac-abb1-0d392f0d7cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:10.827047 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-344fd177-51ee-4fac-abb1-0d392f0d7cf7 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:10.828219 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-344fd177-51ee-4fac-abb1-0d392f0d7cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:10.909631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1079/2189] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:10.917229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4725917-d08b-473a-9185-c822631646c6 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:10.917645 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4725917-d08b-473a-9185-c822631646c6 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:10.917992 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4725917-d08b-473a-9185-c822631646c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:10.942882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-f4725917-d08b-473a-9185-c822631646c6 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:10.945350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-f4725917-d08b-473a-9185-c822631646c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:10.954953 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1111/2190] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:10.962010 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344fd177-51ee-4fac-abb1-0d392f0d7cf7 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:10.970875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344fd177-51ee-4fac-abb1-0d392f0d7cf7 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:10.977931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691385b-3df4-44e6-a283-4a4f24a2eabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:10.978445 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691385b-3df4-44e6-a283-4a4f24a2eabc admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:10.978731 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691385b-3df4-44e6-a283-4a4f24a2eabc admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:10.979072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691385b-3df4-44e6-a283-4a4f24a2eabc admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:10.979731 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691385b-3df4-44e6-a283-4a4f24a2eabc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:47:10.980134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691385b-3df4-44e6-a283-4a4f24a2eabc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:10.981801 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691385b-3df4-44e6-a283-4a4f24a2eabc admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:11.004801 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1080/2191] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:47:11.004910 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4725917-d08b-473a-9185-c822631646c6 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:11.012052 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4725917-d08b-473a-9185-c822631646c6 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:11.017171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea95131-68f7-4507-9ec5-8a9b9cc73e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:11.017824 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea95131-68f7-4507-9ec5-8a9b9cc73e76 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:11.018108 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea95131-68f7-4507-9ec5-8a9b9cc73e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:11.018385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea95131-68f7-4507-9ec5-8a9b9cc73e76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:11.019083 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea95131-68f7-4507-9ec5-8a9b9cc73e76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:11.021107 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea95131-68f7-4507-9ec5-8a9b9cc73e76 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:11.035729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:11.117749 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1112/2192] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:11 2022] POST /identity/v3/projects => generated 407 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:11.122356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691385b-3df4-44e6-a283-4a4f24a2eabc admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:11.142480 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691385b-3df4-44e6-a283-4a4f24a2eabc admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:11.182351 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:11.211103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe5e29-b136-45d3-a716-b918a7a5833a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:11.211821 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe5e29-b136-45d3-a716-b918a7a5833a admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:11.212270 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe5e29-b136-45d3-a716-b918a7a5833a admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:11.212609 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe5e29-b136-45d3-a716-b918a7a5833a admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:11.213356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe5e29-b136-45d3-a716-b918a7a5833a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:11.216052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe5e29-b136-45d3-a716-b918a7a5833a admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:11.308845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1081/2193] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:47:11 2022] POST /identity/v3/users => generated 304 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:11.312740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea95131-68f7-4507-9ec5-8a9b9cc73e76 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:11.324226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea95131-68f7-4507-9ec5-8a9b9cc73e76 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:11.330208 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65a8f1-af28-472f-8319-c53e5104a830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:11.331072 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65a8f1-af28-472f-8319-c53e5104a830 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:11.331431 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65a8f1-af28-472f-8319-c53e5104a830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:11.331721 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65a8f1-af28-472f-8319-c53e5104a830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:11.332468 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65a8f1-af28-472f-8319-c53e5104a830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:11.334197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65a8f1-af28-472f-8319-c53e5104a830 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:11.346708 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1113/2194] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:11.351113 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe5e29-b136-45d3-a716-b918a7a5833a admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:11.361654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe5e29-b136-45d3-a716-b918a7a5833a admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:11.370520 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb639fdc-0e14-4590-9b51-183426ba9260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:11.371138 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb639fdc-0e14-4590-9b51-183426ba9260 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:11.371429 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb639fdc-0e14-4590-9b51-183426ba9260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:11.371711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb639fdc-0e14-4590-9b51-183426ba9260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:11.372446 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb639fdc-0e14-4590-9b51-183426ba9260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:11.374704 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb639fdc-0e14-4590-9b51-183426ba9260 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:11.386748 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1082/2195] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:11.391464 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65a8f1-af28-472f-8319-c53e5104a830 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:11.405791 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65a8f1-af28-472f-8319-c53e5104a830 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:11.414290 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c008787-1eac-4a40-8548-d368bb18b8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:11.415353 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c008787-1eac-4a40-8548-d368bb18b8bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:11.415687 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c008787-1eac-4a40-8548-d368bb18b8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:11.415976 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c008787-1eac-4a40-8548-d368bb18b8bd admin admin] PATH_INFO: `/v3/projects/b0d356ef1611485d8f5412d862ab7ab7/users/d76937f3321b43319c7f2c38e39af79f/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:11.434752 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c008787-1eac-4a40-8548-d368bb18b8bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0d356ef1611485d8f5412d862ab7ab7, user_id=d76937f3321b43319c7f2c38e39af79f, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:11.437710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c008787-1eac-4a40-8548-d368bb18b8bd admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:11.520808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1114/2196] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:11 2022] PUT /identity/v3/projects/b0d356ef1611485d8f5412d862ab7ab7/users/d76937f3321b43319c7f2c38e39af79f/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:11.531989 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb4d7e-b98d-420e-a872-284945d0f1ea None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:11.532286 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb4d7e-b98d-420e-a872-284945d0f1ea None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:11.532570 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb4d7e-b98d-420e-a872-284945d0f1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:11.561391 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-f1fb4d7e-b98d-420e-a872-284945d0f1ea None None] MFA Rules not processed for user `d76937f3321b43319c7f2c38e39af79f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:11.562613 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-f1fb4d7e-b98d-420e-a872-284945d0f1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:11.630036 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1083/2197] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:11 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:11.635163 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c008787-1eac-4a40-8548-d368bb18b8bd admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:11.696705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c008787-1eac-4a40-8548-d368bb18b8bd admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:11.701534 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc3529-0521-41a5-850a-b6cab6d1c2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:11.702200 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc3529-0521-41a5-850a-b6cab6d1c2f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:11.702493 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc3529-0521-41a5-850a-b6cab6d1c2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:11.702777 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc3529-0521-41a5-850a-b6cab6d1c2f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:11.703472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc3529-0521-41a5-850a-b6cab6d1c2f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:11.705432 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc3529-0521-41a5-850a-b6cab6d1c2f9 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:11.800331 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1115/2198] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:11 2022] POST /identity/v3/projects => generated 409 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:11.805392 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb4d7e-b98d-420e-a872-284945d0f1ea None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:11.816350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb4d7e-b98d-420e-a872-284945d0f1ea None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:11.883689 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5346a1-829a-4019-a1b8-4e5a01acb15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:11.883689 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5346a1-829a-4019-a1b8-4e5a01acb15d admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:11.883689 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5346a1-829a-4019-a1b8-4e5a01acb15d admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:11.883689 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5346a1-829a-4019-a1b8-4e5a01acb15d admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:11.884705 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5346a1-829a-4019-a1b8-4e5a01acb15d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:11.885774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5346a1-829a-4019-a1b8-4e5a01acb15d admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:11.966299 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1084/2199] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:47:11 2022] POST /identity/v3/users => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:11.971839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc3529-0521-41a5-850a-b6cab6d1c2f9 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:11.983957 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc3529-0521-41a5-850a-b6cab6d1c2f9 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:11.988929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c56d4-c309-42bb-a186-399061e44560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:11.989638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c56d4-c309-42bb-a186-399061e44560 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:11.989957 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c56d4-c309-42bb-a186-399061e44560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:11.990241 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c56d4-c309-42bb-a186-399061e44560 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:11.991026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c56d4-c309-42bb-a186-399061e44560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:11.993634 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c56d4-c309-42bb-a186-399061e44560 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:12.004636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1116/2200] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:12.009075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5346a1-829a-4019-a1b8-4e5a01acb15d admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:12.019812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5346a1-829a-4019-a1b8-4e5a01acb15d admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:12.024966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256e5cd-59c0-45ae-8208-c593fb6f7b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:12.025873 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256e5cd-59c0-45ae-8208-c593fb6f7b86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:12.026158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256e5cd-59c0-45ae-8208-c593fb6f7b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:12.027118 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256e5cd-59c0-45ae-8208-c593fb6f7b86 admin admin] PATH_INFO: `/v3/projects/74f919a301974ac6b106d13005c0f1fa/users/661fa4b4c4c54fb0a067a396f62d988b/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:12.046426 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256e5cd-59c0-45ae-8208-c593fb6f7b86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74f919a301974ac6b106d13005c0f1fa, user_id=661fa4b4c4c54fb0a067a396f62d988b, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:12.049307 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256e5cd-59c0-45ae-8208-c593fb6f7b86 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:12.130997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1085/2201] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:12 2022] PUT /identity/v3/projects/74f919a301974ac6b106d13005c0f1fa/users/661fa4b4c4c54fb0a067a396f62d988b/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:12.135859 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c56d4-c309-42bb-a186-399061e44560 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:12.146398 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c56d4-c309-42bb-a186-399061e44560 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:12.206555 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2cb73-2d31-4349-bab9-d8befadade24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:12.207236 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2cb73-2d31-4349-bab9-d8befadade24 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:12.207519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2cb73-2d31-4349-bab9-d8befadade24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:12.207798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2cb73-2d31-4349-bab9-d8befadade24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:12.208532 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2cb73-2d31-4349-bab9-d8befadade24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:12.210243 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2cb73-2d31-4349-bab9-d8befadade24 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:12.222121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1117/2202] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:12.226582 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256e5cd-59c0-45ae-8208-c593fb6f7b86 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:12.239167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256e5cd-59c0-45ae-8208-c593fb6f7b86 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:12.245189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b09c24-fb47-44d9-9b9e-37a7119808e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:12.246161 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b09c24-fb47-44d9-9b9e-37a7119808e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:12.246526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b09c24-fb47-44d9-9b9e-37a7119808e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:12.246870 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b09c24-fb47-44d9-9b9e-37a7119808e0 admin admin] PATH_INFO: `/v3/projects/74f919a301974ac6b106d13005c0f1fa/users/661fa4b4c4c54fb0a067a396f62d988b/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:12.248988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b09c24-fb47-44d9-9b9e-37a7119808e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74f919a301974ac6b106d13005c0f1fa, user_id=661fa4b4c4c54fb0a067a396f62d988b, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:12.251703 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b09c24-fb47-44d9-9b9e-37a7119808e0 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:12.258207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:12.263353 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1086/2203] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:12 2022] PUT /identity/v3/projects/74f919a301974ac6b106d13005c0f1fa/users/661fa4b4c4c54fb0a067a396f62d988b/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:12.281312 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e2c9b-96f3-4448-bd3c-b3370ea94c0f None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:12.281683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e2c9b-96f3-4448-bd3c-b3370ea94c0f None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:12.282028 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e2c9b-96f3-4448-bd3c-b3370ea94c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:12.316821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-863e2c9b-96f3-4448-bd3c-b3370ea94c0f None None] MFA Rules not processed for user `661fa4b4c4c54fb0a067a396f62d988b`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:12.318332 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-863e2c9b-96f3-4448-bd3c-b3370ea94c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:12.338602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:12.405226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1118/2204] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:12 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:12.417580 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b09c24-fb47-44d9-9b9e-37a7119808e0 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:12.419330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e1b09c24-fb47-44d9-9b9e-37a7119808e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:12.499690 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b09c24-fb47-44d9-9b9e-37a7119808e0 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:12.503167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930c52d-6006-4ab7-a27b-06ad37612db4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:12.503830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930c52d-6006-4ab7-a27b-06ad37612db4 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:12.504114 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930c52d-6006-4ab7-a27b-06ad37612db4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:12.504418 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930c52d-6006-4ab7-a27b-06ad37612db4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:12.523720 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e930c52d-6006-4ab7-a27b-06ad37612db4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:12.525336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e930c52d-6006-4ab7-a27b-06ad37612db4 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:12.537094 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1087/2205] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:12 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:13.051907 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e2c9b-96f3-4448-bd3c-b3370ea94c0f None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:13.053899 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-863e2c9b-96f3-4448-bd3c-b3370ea94c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:13.067896 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e2c9b-96f3-4448-bd3c-b3370ea94c0f None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:13.071919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cf7b9-0348-4818-bc9f-5a0cb5b4f2ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:13.072609 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cf7b9-0348-4818-bc9f-5a0cb5b4f2ff service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:13.072901 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cf7b9-0348-4818-bc9f-5a0cb5b4f2ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:13.073182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cf7b9-0348-4818-bc9f-5a0cb5b4f2ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:13.141850 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17cf7b9-0348-4818-bc9f-5a0cb5b4f2ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:13.143506 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17cf7b9-0348-4818-bc9f-5a0cb5b4f2ff service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:13.160828 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1119/2206] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:13 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:22.532343 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=27 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:23.151295 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=49 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:23.314627 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930c52d-6006-4ab7-a27b-06ad37612db4 service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:23.453012 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930c52d-6006-4ab7-a27b-06ad37612db4 service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:23.453012 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e33c5-89f5-4ed1-87e3-427d6f5566ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:23.453853 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e33c5-89f5-4ed1-87e3-427d6f5566ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:23.454144 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e33c5-89f5-4ed1-87e3-427d6f5566ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:23.454559 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e33c5-89f5-4ed1-87e3-427d6f5566ef admin admin] PATH_INFO: `/v3/users/d76937f3321b43319c7f2c38e39af79f` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:23.455782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e33c5-89f5-4ed1-87e3-427d6f5566ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=d76937f3321b43319c7f2c38e39af79f)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:23.457875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e33c5-89f5-4ed1-87e3-427d6f5566ef admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:23.947131 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-0c0e33c5-89f5-4ed1-87e3-427d6f5566ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd76937f3321b43319c7f2c38e39af79f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:23.960457 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-0c0e33c5-89f5-4ed1-87e3-427d6f5566ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd76937f3321b43319c7f2c38e39af79f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:24.027588 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-0c0e33c5-89f5-4ed1-87e3-427d6f5566ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd76937f3321b43319c7f2c38e39af79f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:24.028251 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-0c0e33c5-89f5-4ed1-87e3-427d6f5566ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd76937f3321b43319c7f2c38e39af79f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:24.028897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-0c0e33c5-89f5-4ed1-87e3-427d6f5566ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd76937f3321b43319c7f2c38e39af79f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:24.042549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1088/2207] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:23 2022] DELETE /identity/v3/users/d76937f3321b43319c7f2c38e39af79f => generated 0 bytes in 728 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:24.047114 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9c8cd-476f-4756-8ab9-ad96917863af None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:24.047407 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9c8cd-476f-4756-8ab9-ad96917863af None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:24.047687 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9c8cd-476f-4756-8ab9-ad96917863af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:24.101636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-43b9c8cd-476f-4756-8ab9-ad96917863af None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:24.102897 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-43b9c8cd-476f-4756-8ab9-ad96917863af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:24.174523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1120/2208] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:24.188073 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e33c5-89f5-4ed1-87e3-427d6f5566ef admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:24.189645 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-0c0e33c5-89f5-4ed1-87e3-427d6f5566ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:24.236277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:24.296125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e33c5-89f5-4ed1-87e3-427d6f5566ef admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:24.300010 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b16e5-690d-4818-a7f5-75af133cf164 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:24.300693 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b16e5-690d-4818-a7f5-75af133cf164 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:24.300985 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b16e5-690d-4818-a7f5-75af133cf164 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:24.301279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b16e5-690d-4818-a7f5-75af133cf164 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:24.312182 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b16e5-690d-4818-a7f5-75af133cf164 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:24.314353 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b16e5-690d-4818-a7f5-75af133cf164 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:24.325324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1089/2209] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:24.837197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9c8cd-476f-4756-8ab9-ad96917863af None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:24.838687 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-43b9c8cd-476f-4756-8ab9-ad96917863af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:24.853314 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9c8cd-476f-4756-8ab9-ad96917863af None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:24.859323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c23c53-f217-468f-911c-c06398b3b415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:24.860195 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c23c53-f217-468f-911c-c06398b3b415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:24.860528 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c23c53-f217-468f-911c-c06398b3b415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:24.864088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c23c53-f217-468f-911c-c06398b3b415 admin admin] PATH_INFO: `/v3/projects/b0d356ef1611485d8f5412d862ab7ab7` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:24.864088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c23c53-f217-468f-911c-c06398b3b415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0d356ef1611485d8f5412d862ab7ab7)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:24.864088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c23c53-f217-468f-911c-c06398b3b415 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:25.018719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-70c23c53-f217-468f-911c-c06398b3b415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0d356ef1611485d8f5412d862ab7ab7', 'request_id': 'req-70c23c53-f217-468f-911c-c06398b3b415'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:25.028262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-70c23c53-f217-468f-911c-c06398b3b415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0d356ef1611485d8f5412d862ab7ab7', 'request_id': 'req-70c23c53-f217-468f-911c-c06398b3b415'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:25.034893 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-70c23c53-f217-468f-911c-c06398b3b415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0d356ef1611485d8f5412d862ab7ab7', 'request_id': 'req-70c23c53-f217-468f-911c-c06398b3b415'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:25.125099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-70c23c53-f217-468f-911c-c06398b3b415 admin admin] The token cache is being invalidate because project b0d356ef1611485d8f5412d862ab7ab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:25.125681 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-70c23c53-f217-468f-911c-c06398b3b415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:25.126350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-70c23c53-f217-468f-911c-c06398b3b415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:25.127992 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1121/2210] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:24 2022] DELETE /identity/v3/projects/b0d356ef1611485d8f5412d862ab7ab7 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:25.132079 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b16e5-690d-4818-a7f5-75af133cf164 service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:25.143692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b16e5-690d-4818-a7f5-75af133cf164 service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:25.218577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92051e-49df-4156-819d-ff11b52837cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:25.219466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92051e-49df-4156-819d-ff11b52837cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:25.219750 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92051e-49df-4156-819d-ff11b52837cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:25.220077 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92051e-49df-4156-819d-ff11b52837cd admin admin] PATH_INFO: `/v3/users/661fa4b4c4c54fb0a067a396f62d988b` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:25.222021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92051e-49df-4156-819d-ff11b52837cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=661fa4b4c4c54fb0a067a396f62d988b)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:25.224172 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92051e-49df-4156-819d-ff11b52837cd admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:25.482148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fb92051e-49df-4156-819d-ff11b52837cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '661fa4b4c4c54fb0a067a396f62d988b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:25.497508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fb92051e-49df-4156-819d-ff11b52837cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '661fa4b4c4c54fb0a067a396f62d988b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:25.610316 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fb92051e-49df-4156-819d-ff11b52837cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '661fa4b4c4c54fb0a067a396f62d988b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:25.611045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fb92051e-49df-4156-819d-ff11b52837cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '661fa4b4c4c54fb0a067a396f62d988b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:25.611707 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fb92051e-49df-4156-819d-ff11b52837cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '661fa4b4c4c54fb0a067a396f62d988b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:25.626190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1090/2211] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:25 2022] DELETE /identity/v3/users/661fa4b4c4c54fb0a067a396f62d988b => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:26.181754 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c23c53-f217-468f-911c-c06398b3b415 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:26.183405 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-70c23c53-f217-468f-911c-c06398b3b415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:26.197415 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c23c53-f217-468f-911c-c06398b3b415 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:26.293055 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:26.298761 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae37220-c230-4e17-a83d-872ca1927f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:26.299881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae37220-c230-4e17-a83d-872ca1927f1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:26.300168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae37220-c230-4e17-a83d-872ca1927f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:26.300442 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae37220-c230-4e17-a83d-872ca1927f1a admin admin] PATH_INFO: `/v3/projects/74f919a301974ac6b106d13005c0f1fa` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:26.301812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae37220-c230-4e17-a83d-872ca1927f1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=74f919a301974ac6b106d13005c0f1fa)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:26.304460 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae37220-c230-4e17-a83d-872ca1927f1a admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:26.418424 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3ae37220-c230-4e17-a83d-872ca1927f1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74f919a301974ac6b106d13005c0f1fa', 'request_id': 'req-3ae37220-c230-4e17-a83d-872ca1927f1a'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:26.425024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3ae37220-c230-4e17-a83d-872ca1927f1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74f919a301974ac6b106d13005c0f1fa', 'request_id': 'req-3ae37220-c230-4e17-a83d-872ca1927f1a'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:26.431699 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3ae37220-c230-4e17-a83d-872ca1927f1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74f919a301974ac6b106d13005c0f1fa', 'request_id': 'req-3ae37220-c230-4e17-a83d-872ca1927f1a'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:26.472920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3ae37220-c230-4e17-a83d-872ca1927f1a admin admin] The token cache is being invalidate because project 74f919a301974ac6b106d13005c0f1fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:26.473225 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3ae37220-c230-4e17-a83d-872ca1927f1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:26.473886 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3ae37220-c230-4e17-a83d-872ca1927f1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:26.476541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1122/2212] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:26 2022] DELETE /identity/v3/projects/74f919a301974ac6b106d13005c0f1fa => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:26.481832 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555438b6-aca8-484b-86bf-bcb0d0a63c97 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:26.482318 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555438b6-aca8-484b-86bf-bcb0d0a63c97 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:26.482426 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555438b6-aca8-484b-86bf-bcb0d0a63c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:26.524153 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-555438b6-aca8-484b-86bf-bcb0d0a63c97 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:26.528693 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-555438b6-aca8-484b-86bf-bcb0d0a63c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:26.529706 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:26.559715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99a69e-20a6-4a1f-ac23-5a1d041c0f2d None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:26.560002 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99a69e-20a6-4a1f-ac23-5a1d041c0f2d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:26.560280 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99a69e-20a6-4a1f-ac23-5a1d041c0f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:26.581885 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-7b99a69e-20a6-4a1f-ac23-5a1d041c0f2d None None] MFA Rules not processed for user `1d7e431402f8433fb7565f69103485ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:26.583110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-7b99a69e-20a6-4a1f-ac23-5a1d041c0f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:26.587848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1091/2213] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:26.596395 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fc933-c653-418c-ab5a-a4afb1c43970 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:26.596724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fc933-c653-418c-ab5a-a4afb1c43970 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:26.597002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fc933-c653-418c-ab5a-a4afb1c43970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:26.617469 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-091fc933-c653-418c-ab5a-a4afb1c43970 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:26.618932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-091fc933-c653-418c-ab5a-a4afb1c43970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:26.626927 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1092/2214] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:47:26.631081 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fc933-c653-418c-ab5a-a4afb1c43970 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:26.640824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fc933-c653-418c-ab5a-a4afb1c43970 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:26.645377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d537e8-8ddf-4f74-bfc9-b7f85c3ceee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:26.645508 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1123/2215] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:47:26.649284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d537e8-8ddf-4f74-bfc9-b7f85c3ceee5 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:26.649284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d537e8-8ddf-4f74-bfc9-b7f85c3ceee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:26.649284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d537e8-8ddf-4f74-bfc9-b7f85c3ceee5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:26.649284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d537e8-8ddf-4f74-bfc9-b7f85c3ceee5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:47:26.649284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d537e8-8ddf-4f74-bfc9-b7f85c3ceee5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:26.649833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d537e8-8ddf-4f74-bfc9-b7f85c3ceee5 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:26.661637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1093/2216] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:47:26.666696 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99a69e-20a6-4a1f-ac23-5a1d041c0f2d None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:26.677661 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99a69e-20a6-4a1f-ac23-5a1d041c0f2d None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:26.682548 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1266b-184c-4c42-9df4-9b3fca05b6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:26.683229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1266b-184c-4c42-9df4-9b3fca05b6d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:26.683511 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1266b-184c-4c42-9df4-9b3fca05b6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:26.683786 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1266b-184c-4c42-9df4-9b3fca05b6d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:26.684471 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1266b-184c-4c42-9df4-9b3fca05b6d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:26.686443 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1266b-184c-4c42-9df4-9b3fca05b6d0 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:26.761008 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1124/2217] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:26 2022] POST /identity/v3/projects => generated 367 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:26.765330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d537e8-8ddf-4f74-bfc9-b7f85c3ceee5 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:26.776284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d537e8-8ddf-4f74-bfc9-b7f85c3ceee5 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:26.832375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b97c26-a687-4f69-b1aa-549121f15b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:26.833103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b97c26-a687-4f69-b1aa-549121f15b75 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:26.833389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b97c26-a687-4f69-b1aa-549121f15b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:26.833665 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b97c26-a687-4f69-b1aa-549121f15b75 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:26.835026 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b97c26-a687-4f69-b1aa-549121f15b75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:26.837025 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b97c26-a687-4f69-b1aa-549121f15b75 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:26.907460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1094/2218] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:47:26 2022] POST /identity/v3/users => generated 284 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:26.911637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1266b-184c-4c42-9df4-9b3fca05b6d0 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:26.921834 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1266b-184c-4c42-9df4-9b3fca05b6d0 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:26.926410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232110b-7e7f-4fda-8896-d57cc83d2d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:26.927063 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232110b-7e7f-4fda-8896-d57cc83d2d7e admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:26.927347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232110b-7e7f-4fda-8896-d57cc83d2d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:26.927632 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232110b-7e7f-4fda-8896-d57cc83d2d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:26.928654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232110b-7e7f-4fda-8896-d57cc83d2d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:26.930241 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232110b-7e7f-4fda-8896-d57cc83d2d7e admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:26.941562 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1125/2219] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:26.946164 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b97c26-a687-4f69-b1aa-549121f15b75 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:26.956321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b97c26-a687-4f69-b1aa-549121f15b75 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:26.961481 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d118e1f3-42db-4763-ba25-2df0b9bf8ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:26.962134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118e1f3-42db-4763-ba25-2df0b9bf8ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:26.962422 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118e1f3-42db-4763-ba25-2df0b9bf8ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:26.962703 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118e1f3-42db-4763-ba25-2df0b9bf8ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:26.963434 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d118e1f3-42db-4763-ba25-2df0b9bf8ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:26.965084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d118e1f3-42db-4763-ba25-2df0b9bf8ea1 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:26.975107 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1095/2220] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:26.978808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232110b-7e7f-4fda-8896-d57cc83d2d7e admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:27.002065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232110b-7e7f-4fda-8896-d57cc83d2d7e admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:27.016108 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063bce6-94c7-410b-8853-dec42a0e6f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:27.016108 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063bce6-94c7-410b-8853-dec42a0e6f1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:27.016108 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063bce6-94c7-410b-8853-dec42a0e6f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:27.016108 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063bce6-94c7-410b-8853-dec42a0e6f1b admin admin] PATH_INFO: `/v3/projects/45fe67e4960446f99bbf62b6fbdf16fa/users/f5db2b27094040ba9d0ccccaf764aa13/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:27.041054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063bce6-94c7-410b-8853-dec42a0e6f1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=45fe67e4960446f99bbf62b6fbdf16fa, user_id=f5db2b27094040ba9d0ccccaf764aa13, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:27.043202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063bce6-94c7-410b-8853-dec42a0e6f1b admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:27.149824 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1126/2221] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:26 2022] PUT /identity/v3/projects/45fe67e4960446f99bbf62b6fbdf16fa/users/f5db2b27094040ba9d0ccccaf764aa13/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:27.160243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52217f7b-b518-4c56-afee-6bab07cf195a None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:27.160545 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52217f7b-b518-4c56-afee-6bab07cf195a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:27.160860 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52217f7b-b518-4c56-afee-6bab07cf195a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:27.188983 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-52217f7b-b518-4c56-afee-6bab07cf195a None None] MFA Rules not processed for user `f5db2b27094040ba9d0ccccaf764aa13`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:27.190343 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-52217f7b-b518-4c56-afee-6bab07cf195a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:27.258674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:27.264092 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1096/2222] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:27 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:27.268707 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063bce6-94c7-410b-8853-dec42a0e6f1b admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:27.313950 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:27.337180 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063bce6-94c7-410b-8853-dec42a0e6f1b admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:27.342517 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972297e6-21c4-406f-8fbc-d0b741920642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:27.345427 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972297e6-21c4-406f-8fbc-d0b741920642 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:27.345427 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972297e6-21c4-406f-8fbc-d0b741920642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:27.345427 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972297e6-21c4-406f-8fbc-d0b741920642 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:27.345427 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972297e6-21c4-406f-8fbc-d0b741920642 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:27.346348 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972297e6-21c4-406f-8fbc-d0b741920642 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:27.428485 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1127/2223] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:27 2022] POST /identity/v3/projects => generated 367 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:27.432519 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52217f7b-b518-4c56-afee-6bab07cf195a None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:27.443719 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52217f7b-b518-4c56-afee-6bab07cf195a None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:27.504507 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2599c8-04ad-411a-a710-31f91a6c6d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:27.505242 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2599c8-04ad-411a-a710-31f91a6c6d5c admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:27.505502 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2599c8-04ad-411a-a710-31f91a6c6d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:27.506321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2599c8-04ad-411a-a710-31f91a6c6d5c admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:27.507523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2599c8-04ad-411a-a710-31f91a6c6d5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:27.509506 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2599c8-04ad-411a-a710-31f91a6c6d5c admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:27.633825 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1097/2224] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:47:27 2022] POST /identity/v3/users => generated 290 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:27.638148 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972297e6-21c4-406f-8fbc-d0b741920642 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:27.649518 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972297e6-21c4-406f-8fbc-d0b741920642 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:27.654043 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00cbeb-2237-45cb-8b19-652a47fa3c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:27.654753 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00cbeb-2237-45cb-8b19-652a47fa3c2b admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:27.655065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00cbeb-2237-45cb-8b19-652a47fa3c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:27.655345 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00cbeb-2237-45cb-8b19-652a47fa3c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:27.656072 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00cbeb-2237-45cb-8b19-652a47fa3c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:27.657690 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00cbeb-2237-45cb-8b19-652a47fa3c2b admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:27.669126 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1128/2225] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:27.673273 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2599c8-04ad-411a-a710-31f91a6c6d5c admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:27.683492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2599c8-04ad-411a-a710-31f91a6c6d5c admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:27.688143 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49492476-947f-46cb-9eb5-9362e8eb67b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:27.689058 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49492476-947f-46cb-9eb5-9362e8eb67b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:27.689343 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49492476-947f-46cb-9eb5-9362e8eb67b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:27.689619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49492476-947f-46cb-9eb5-9362e8eb67b0 admin admin] PATH_INFO: `/v3/projects/662b069474474c2d980ca5ce299e2b7f/users/474023caa8c6484aabbb80bf21eee0fb/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:27.706586 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49492476-947f-46cb-9eb5-9362e8eb67b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=662b069474474c2d980ca5ce299e2b7f, user_id=474023caa8c6484aabbb80bf21eee0fb, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:27.710010 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49492476-947f-46cb-9eb5-9362e8eb67b0 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:27.788215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1098/2226] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:27 2022] PUT /identity/v3/projects/662b069474474c2d980ca5ce299e2b7f/users/474023caa8c6484aabbb80bf21eee0fb/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:27.792230 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00cbeb-2237-45cb-8b19-652a47fa3c2b admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:27.802315 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00cbeb-2237-45cb-8b19-652a47fa3c2b admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:27.857792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129b4b7-37cb-4ef0-a396-b923ec1c7f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:27.858450 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129b4b7-37cb-4ef0-a396-b923ec1c7f09 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:27.858738 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129b4b7-37cb-4ef0-a396-b923ec1c7f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:27.859017 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129b4b7-37cb-4ef0-a396-b923ec1c7f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:27.859739 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c129b4b7-37cb-4ef0-a396-b923ec1c7f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:27.861368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c129b4b7-37cb-4ef0-a396-b923ec1c7f09 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:27.872493 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1129/2227] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:47:27.876745 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49492476-947f-46cb-9eb5-9362e8eb67b0 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:27.889571 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49492476-947f-46cb-9eb5-9362e8eb67b0 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:27.894195 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79a691-b273-4608-a8bc-9b250dbedabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:27.895149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79a691-b273-4608-a8bc-9b250dbedabb admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:27.895428 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79a691-b273-4608-a8bc-9b250dbedabb admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:27.895706 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79a691-b273-4608-a8bc-9b250dbedabb admin admin] PATH_INFO: `/v3/projects/662b069474474c2d980ca5ce299e2b7f/users/474023caa8c6484aabbb80bf21eee0fb/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:27.897990 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79a691-b273-4608-a8bc-9b250dbedabb admin admin] RBAC: Authorizing `identity:create_grant(project_id=662b069474474c2d980ca5ce299e2b7f, user_id=474023caa8c6484aabbb80bf21eee0fb, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:27.900692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79a691-b273-4608-a8bc-9b250dbedabb admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:27.910963 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1099/2228] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:27 2022] PUT /identity/v3/projects/662b069474474c2d980ca5ce299e2b7f/users/474023caa8c6484aabbb80bf21eee0fb/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:27.921564 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48197fcd-3c50-4539-af90-5b4dcf8f741f None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:27.921859 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48197fcd-3c50-4539-af90-5b4dcf8f741f None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:27.922140 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48197fcd-3c50-4539-af90-5b4dcf8f741f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:27.949432 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-48197fcd-3c50-4539-af90-5b4dcf8f741f None None] MFA Rules not processed for user `474023caa8c6484aabbb80bf21eee0fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:27.950650 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-48197fcd-3c50-4539-af90-5b4dcf8f741f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:28.025440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1130/2229] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:27 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:28.036685 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79a691-b273-4608-a8bc-9b250dbedabb admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:28.038940 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-1d79a691-b273-4608-a8bc-9b250dbedabb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:28.105097 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79a691-b273-4608-a8bc-9b250dbedabb admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:28.109666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24a05b-cacd-4d66-8895-3a3a7438f4f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:28.110443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24a05b-cacd-4d66-8895-3a3a7438f4f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:28.110791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24a05b-cacd-4d66-8895-3a3a7438f4f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:28.111151 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24a05b-cacd-4d66-8895-3a3a7438f4f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:28.125544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24a05b-cacd-4d66-8895-3a3a7438f4f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:28.127031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24a05b-cacd-4d66-8895-3a3a7438f4f4 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:28.137558 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1100/2230] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:28 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:28.390644 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48197fcd-3c50-4539-af90-5b4dcf8f741f None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:28.401855 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48197fcd-3c50-4539-af90-5b4dcf8f741f None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:28.405477 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d7cce-5e46-4dd5-b71d-de7e705a6345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:28.406143 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d7cce-5e46-4dd5-b71d-de7e705a6345 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:28.406411 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d7cce-5e46-4dd5-b71d-de7e705a6345 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:28.406691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d7cce-5e46-4dd5-b71d-de7e705a6345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:28.435840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:28.465023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d7cce-5e46-4dd5-b71d-de7e705a6345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:28.466528 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d7cce-5e46-4dd5-b71d-de7e705a6345 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:28.476246 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1131/2231] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:28 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:30.977277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24a05b-cacd-4d66-8895-3a3a7438f4f4 service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:30.978711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-9f24a05b-cacd-4d66-8895-3a3a7438f4f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:31.035461 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24a05b-cacd-4d66-8895-3a3a7438f4f4 service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:31.040009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cb1eb-3b53-4608-a416-2c11e57f6a26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:31.040750 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cb1eb-3b53-4608-a416-2c11e57f6a26 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:31.041151 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cb1eb-3b53-4608-a416-2c11e57f6a26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:31.041433 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cb1eb-3b53-4608-a416-2c11e57f6a26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:31.043994 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-884cb1eb-3b53-4608-a416-2c11e57f6a26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:31.051102 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:31.111271 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cb1eb-3b53-4608-a416-2c11e57f6a26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:31.112742 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cb1eb-3b53-4608-a416-2c11e57f6a26 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:31.122572 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1101/2232] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:35.002244 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d7cce-5e46-4dd5-b71d-de7e705a6345 service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:35.065500 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d7cce-5e46-4dd5-b71d-de7e705a6345 service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:35.070786 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116c615-a6d6-4895-80b7-f888d359acbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:35.071665 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116c615-a6d6-4895-80b7-f888d359acbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:35.071956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116c615-a6d6-4895-80b7-f888d359acbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:35.072266 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116c615-a6d6-4895-80b7-f888d359acbf admin admin] PATH_INFO: `/v3/users/f5db2b27094040ba9d0ccccaf764aa13` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:35.073510 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116c615-a6d6-4895-80b7-f888d359acbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5db2b27094040ba9d0ccccaf764aa13)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:35.075430 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116c615-a6d6-4895-80b7-f888d359acbf admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:35.270347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a116c615-a6d6-4895-80b7-f888d359acbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5db2b27094040ba9d0ccccaf764aa13'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:35.284039 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a116c615-a6d6-4895-80b7-f888d359acbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5db2b27094040ba9d0ccccaf764aa13'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:35.284823 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a116c615-a6d6-4895-80b7-f888d359acbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5db2b27094040ba9d0ccccaf764aa13'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:35.285549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a116c615-a6d6-4895-80b7-f888d359acbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5db2b27094040ba9d0ccccaf764aa13'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:35.293389 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a116c615-a6d6-4895-80b7-f888d359acbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5db2b27094040ba9d0ccccaf764aa13'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:35.375839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1132/2233] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:35 2022] DELETE /identity/v3/users/f5db2b27094040ba9d0ccccaf764aa13 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:35.380672 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49a71a-0b09-4136-941b-7095922a71d5 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:35.381149 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49a71a-0b09-4136-941b-7095922a71d5 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:35.381232 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49a71a-0b09-4136-941b-7095922a71d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:35.402847 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-9f49a71a-0b09-4136-941b-7095922a71d5 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:35.404364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-9f49a71a-0b09-4136-941b-7095922a71d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:35.471604 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1102/2234] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:35.481840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116c615-a6d6-4895-80b7-f888d359acbf admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:35.483290 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a116c615-a6d6-4895-80b7-f888d359acbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:35.550689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:35.555033 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116c615-a6d6-4895-80b7-f888d359acbf admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:35.558967 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50bb35-7eef-4138-a8b5-01e7c55fa21f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:35.559903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50bb35-7eef-4138-a8b5-01e7c55fa21f service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:35.559903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50bb35-7eef-4138-a8b5-01e7c55fa21f service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:35.560178 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50bb35-7eef-4138-a8b5-01e7c55fa21f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:35.573129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50bb35-7eef-4138-a8b5-01e7c55fa21f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:35.574651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50bb35-7eef-4138-a8b5-01e7c55fa21f service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:35.587636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1133/2235] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:36.380445 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49a71a-0b09-4136-941b-7095922a71d5 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:36.383165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-9f49a71a-0b09-4136-941b-7095922a71d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:36.396878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49a71a-0b09-4136-941b-7095922a71d5 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:36.402829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026df045-9ecb-4b1f-af9a-59c73de48c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:36.403895 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026df045-9ecb-4b1f-af9a-59c73de48c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:36.404179 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026df045-9ecb-4b1f-af9a-59c73de48c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:36.404475 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026df045-9ecb-4b1f-af9a-59c73de48c2a admin admin] PATH_INFO: `/v3/projects/45fe67e4960446f99bbf62b6fbdf16fa` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:36.405846 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026df045-9ecb-4b1f-af9a-59c73de48c2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=45fe67e4960446f99bbf62b6fbdf16fa)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:36.408030 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026df045-9ecb-4b1f-af9a-59c73de48c2a admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:36.553716 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-026df045-9ecb-4b1f-af9a-59c73de48c2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45fe67e4960446f99bbf62b6fbdf16fa', 'request_id': 'req-026df045-9ecb-4b1f-af9a-59c73de48c2a'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:36.644229 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-026df045-9ecb-4b1f-af9a-59c73de48c2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45fe67e4960446f99bbf62b6fbdf16fa', 'request_id': 'req-026df045-9ecb-4b1f-af9a-59c73de48c2a'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:36.652098 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-026df045-9ecb-4b1f-af9a-59c73de48c2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45fe67e4960446f99bbf62b6fbdf16fa', 'request_id': 'req-026df045-9ecb-4b1f-af9a-59c73de48c2a'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:36.673916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-026df045-9ecb-4b1f-af9a-59c73de48c2a admin admin] The token cache is being invalidate because project 45fe67e4960446f99bbf62b6fbdf16fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:36.674219 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-026df045-9ecb-4b1f-af9a-59c73de48c2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:36.675221 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-026df045-9ecb-4b1f-af9a-59c73de48c2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:36.685250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1103/2236] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:36 2022] DELETE /identity/v3/projects/45fe67e4960446f99bbf62b6fbdf16fa => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:36.686518 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50bb35-7eef-4138-a8b5-01e7c55fa21f service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:36.702729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50bb35-7eef-4138-a8b5-01e7c55fa21f service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:36.778343 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358688be-4092-4b0a-9265-20963ce82b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:36.780182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358688be-4092-4b0a-9265-20963ce82b86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:36.780537 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358688be-4092-4b0a-9265-20963ce82b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:36.781767 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358688be-4092-4b0a-9265-20963ce82b86 admin admin] PATH_INFO: `/v3/users/474023caa8c6484aabbb80bf21eee0fb` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:36.783188 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358688be-4092-4b0a-9265-20963ce82b86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=474023caa8c6484aabbb80bf21eee0fb)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:36.785297 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358688be-4092-4b0a-9265-20963ce82b86 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:36.982169 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-358688be-4092-4b0a-9265-20963ce82b86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '474023caa8c6484aabbb80bf21eee0fb'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:37.011483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-358688be-4092-4b0a-9265-20963ce82b86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '474023caa8c6484aabbb80bf21eee0fb'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:37.012106 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-358688be-4092-4b0a-9265-20963ce82b86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '474023caa8c6484aabbb80bf21eee0fb'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:37.012944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-358688be-4092-4b0a-9265-20963ce82b86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '474023caa8c6484aabbb80bf21eee0fb'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:37.025314 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-358688be-4092-4b0a-9265-20963ce82b86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '474023caa8c6484aabbb80bf21eee0fb'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:37.089782 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1134/2237] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:36 2022] DELETE /identity/v3/users/474023caa8c6484aabbb80bf21eee0fb => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:37.480827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026df045-9ecb-4b1f-af9a-59c73de48c2a admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:37.483435 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-026df045-9ecb-4b1f-af9a-59c73de48c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:37.518325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026df045-9ecb-4b1f-af9a-59c73de48c2a admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:37.552964 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:37.620925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96768b4-4110-45e3-b4c1-f31691cf5f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:37.621824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96768b4-4110-45e3-b4c1-f31691cf5f81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:37.622111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96768b4-4110-45e3-b4c1-f31691cf5f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:37.622390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96768b4-4110-45e3-b4c1-f31691cf5f81 admin admin] PATH_INFO: `/v3/projects/662b069474474c2d980ca5ce299e2b7f` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:37.626665 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96768b4-4110-45e3-b4c1-f31691cf5f81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=662b069474474c2d980ca5ce299e2b7f)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:37.629315 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96768b4-4110-45e3-b4c1-f31691cf5f81 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:37.776891 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f96768b4-4110-45e3-b4c1-f31691cf5f81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '662b069474474c2d980ca5ce299e2b7f', 'request_id': 'req-f96768b4-4110-45e3-b4c1-f31691cf5f81'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:37.860715 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f96768b4-4110-45e3-b4c1-f31691cf5f81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '662b069474474c2d980ca5ce299e2b7f', 'request_id': 'req-f96768b4-4110-45e3-b4c1-f31691cf5f81'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:37.867700 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f96768b4-4110-45e3-b4c1-f31691cf5f81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '662b069474474c2d980ca5ce299e2b7f', 'request_id': 'req-f96768b4-4110-45e3-b4c1-f31691cf5f81'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:37.879200 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f96768b4-4110-45e3-b4c1-f31691cf5f81 admin admin] The token cache is being invalidate because project 662b069474474c2d980ca5ce299e2b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:37.879495 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f96768b4-4110-45e3-b4c1-f31691cf5f81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:37.880145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f96768b4-4110-45e3-b4c1-f31691cf5f81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:37.885495 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1104/2238] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:37 2022] DELETE /identity/v3/projects/662b069474474c2d980ca5ce299e2b7f => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:37.887413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ee9f5-d670-46b7-b734-186d630d9861 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:37.892409 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ee9f5-d670-46b7-b734-186d630d9861 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:37.894705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ee9f5-d670-46b7-b734-186d630d9861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:37.965160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-7e6ee9f5-d670-46b7-b734-186d630d9861 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:37.966714 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-7e6ee9f5-d670-46b7-b734-186d630d9861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:38.045235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1135/2239] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:38.053654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e16dd-bb57-443b-8005-0352f57a59da None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:38.054130 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e16dd-bb57-443b-8005-0352f57a59da None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:38.054412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e16dd-bb57-443b-8005-0352f57a59da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:38.095513 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-991e16dd-bb57-443b-8005-0352f57a59da None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:38.096768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-991e16dd-bb57-443b-8005-0352f57a59da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:38.115627 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1105/2240] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:38.116198 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ee9f5-d670-46b7-b734-186d630d9861 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:38.121636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:38.127171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ee9f5-d670-46b7-b734-186d630d9861 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:38.132826 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9799627-bed3-4c66-a7b6-b126cc3851fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:38.133511 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9799627-bed3-4c66-a7b6-b126cc3851fa admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:38.133826 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9799627-bed3-4c66-a7b6-b126cc3851fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:38.134104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9799627-bed3-4c66-a7b6-b126cc3851fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:38.134851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9799627-bed3-4c66-a7b6-b126cc3851fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:47:38.135195 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9799627-bed3-4c66-a7b6-b126cc3851fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:38.136821 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9799627-bed3-4c66-a7b6-b126cc3851fa admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:38.150116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1136/2241] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:47:38.157554 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e16dd-bb57-443b-8005-0352f57a59da None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:38.168019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e16dd-bb57-443b-8005-0352f57a59da None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:38.173533 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a39b7-5eca-46b8-a56e-168ceb5f86f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:38.174381 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a39b7-5eca-46b8-a56e-168ceb5f86f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:38.174950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a39b7-5eca-46b8-a56e-168ceb5f86f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:38.175391 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a39b7-5eca-46b8-a56e-168ceb5f86f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:38.176096 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a39b7-5eca-46b8-a56e-168ceb5f86f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:38.178704 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a39b7-5eca-46b8-a56e-168ceb5f86f5 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:38.285844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1106/2242] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:38 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:38.291068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9799627-bed3-4c66-a7b6-b126cc3851fa admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:38.304242 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9799627-bed3-4c66-a7b6-b126cc3851fa admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:38.363902 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913df6a1-32ae-4ebc-8113-3a8cc73fd4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:38.364599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913df6a1-32ae-4ebc-8113-3a8cc73fd4e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:38.364913 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913df6a1-32ae-4ebc-8113-3a8cc73fd4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:38.365192 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913df6a1-32ae-4ebc-8113-3a8cc73fd4e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:38.366147 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913df6a1-32ae-4ebc-8113-3a8cc73fd4e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:38.368331 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913df6a1-32ae-4ebc-8113-3a8cc73fd4e8 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:38.464629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1137/2243] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:47:38 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:38.466653 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a39b7-5eca-46b8-a56e-168ceb5f86f5 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:38.478386 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a39b7-5eca-46b8-a56e-168ceb5f86f5 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:38.485814 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3831b73-138b-4795-ae25-49844c20dc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:38.485814 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3831b73-138b-4795-ae25-49844c20dc01 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:38.485814 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3831b73-138b-4795-ae25-49844c20dc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:38.485814 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3831b73-138b-4795-ae25-49844c20dc01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:38.486628 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3831b73-138b-4795-ae25-49844c20dc01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:38.486628 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3831b73-138b-4795-ae25-49844c20dc01 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:38.498381 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1107/2244] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:38.502255 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913df6a1-32ae-4ebc-8113-3a8cc73fd4e8 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:38.515816 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913df6a1-32ae-4ebc-8113-3a8cc73fd4e8 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:38.524099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f30ee-215b-4b18-89b5-166d0318af3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:38.525375 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f30ee-215b-4b18-89b5-166d0318af3a admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:38.526064 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f30ee-215b-4b18-89b5-166d0318af3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:38.526362 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f30ee-215b-4b18-89b5-166d0318af3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:38.527719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f30ee-215b-4b18-89b5-166d0318af3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:38.530743 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f30ee-215b-4b18-89b5-166d0318af3a admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:38.543842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1138/2245] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:38.548797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3831b73-138b-4795-ae25-49844c20dc01 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:38.559272 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3831b73-138b-4795-ae25-49844c20dc01 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:38.563983 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225845b-787b-4132-b142-b5022b5cc554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:38.566254 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225845b-787b-4132-b142-b5022b5cc554 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:38.566254 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225845b-787b-4132-b142-b5022b5cc554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:38.566254 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225845b-787b-4132-b142-b5022b5cc554 admin admin] PATH_INFO: `/v3/projects/5c5a84e8906e4fa0b7175ead89168d4a/users/c81ae9cec9d64bc993df2bc89374158f/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:38.580109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225845b-787b-4132-b142-b5022b5cc554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c5a84e8906e4fa0b7175ead89168d4a, user_id=c81ae9cec9d64bc993df2bc89374158f, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:38.582797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225845b-787b-4132-b142-b5022b5cc554 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:38.678241 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1108/2246] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:38 2022] PUT /identity/v3/projects/5c5a84e8906e4fa0b7175ead89168d4a/users/c81ae9cec9d64bc993df2bc89374158f/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:38.689067 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198cdaa-9538-4a4b-a7c0-67763e6d644c None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:38.689448 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198cdaa-9538-4a4b-a7c0-67763e6d644c None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:38.690007 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198cdaa-9538-4a4b-a7c0-67763e6d644c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:38.719657 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-0198cdaa-9538-4a4b-a7c0-67763e6d644c None None] MFA Rules not processed for user `c81ae9cec9d64bc993df2bc89374158f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:38.720890 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-0198cdaa-9538-4a4b-a7c0-67763e6d644c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:38.793273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1139/2247] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:38.797997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225845b-787b-4132-b142-b5022b5cc554 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:38.855449 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:38.870752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225845b-787b-4132-b142-b5022b5cc554 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:38.875455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516dd36d-8034-44fe-bc22-9714f21deb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:38.876114 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516dd36d-8034-44fe-bc22-9714f21deb03 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:38.876400 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516dd36d-8034-44fe-bc22-9714f21deb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:38.879052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516dd36d-8034-44fe-bc22-9714f21deb03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:38.879052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516dd36d-8034-44fe-bc22-9714f21deb03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:38.879335 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516dd36d-8034-44fe-bc22-9714f21deb03 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:38.970513 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1109/2248] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:38 2022] POST /identity/v3/projects => generated 387 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:47:38.974622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198cdaa-9538-4a4b-a7c0-67763e6d644c None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:38.985333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198cdaa-9538-4a4b-a7c0-67763e6d644c None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:39.036116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:39.042149 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e968ce1-303e-45ee-95e7-f22e0c796dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:39.042819 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e968ce1-303e-45ee-95e7-f22e0c796dac admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:39.043104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e968ce1-303e-45ee-95e7-f22e0c796dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:39.043382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e968ce1-303e-45ee-95e7-f22e0c796dac admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:39.044064 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e968ce1-303e-45ee-95e7-f22e0c796dac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:39.046040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e968ce1-303e-45ee-95e7-f22e0c796dac admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:39.129895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1140/2249] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:47:38 2022] POST /identity/v3/users => generated 300 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:39.134602 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516dd36d-8034-44fe-bc22-9714f21deb03 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:39.145670 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516dd36d-8034-44fe-bc22-9714f21deb03 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:39.150446 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c5914-1163-40e6-978a-ecf744667cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:39.151130 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c5914-1163-40e6-978a-ecf744667cbb admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:39.151416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c5914-1163-40e6-978a-ecf744667cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:39.151695 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c5914-1163-40e6-978a-ecf744667cbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:39.152426 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c5914-1163-40e6-978a-ecf744667cbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:39.154125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c5914-1163-40e6-978a-ecf744667cbb admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:39.165777 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1110/2250] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:47:39.169995 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e968ce1-303e-45ee-95e7-f22e0c796dac admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:39.180289 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e968ce1-303e-45ee-95e7-f22e0c796dac admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:39.184869 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f6005-def6-4968-81a2-d6327c150ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:39.185741 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f6005-def6-4968-81a2-d6327c150ffd admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:39.186025 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f6005-def6-4968-81a2-d6327c150ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:39.186303 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f6005-def6-4968-81a2-d6327c150ffd admin admin] PATH_INFO: `/v3/projects/936f4081e609406b8d5214fe82223a0c/users/37881f4e58254ba4840d4ea1f9a5b163/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:39.201704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f6005-def6-4968-81a2-d6327c150ffd admin admin] RBAC: Authorizing `identity:create_grant(project_id=936f4081e609406b8d5214fe82223a0c, user_id=37881f4e58254ba4840d4ea1f9a5b163, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:39.204440 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f6005-def6-4968-81a2-d6327c150ffd admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:39.262167 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1141/2251] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:39 2022] PUT /identity/v3/projects/936f4081e609406b8d5214fe82223a0c/users/37881f4e58254ba4840d4ea1f9a5b163/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:39.266033 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c5914-1163-40e6-978a-ecf744667cbb admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:39.278558 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c5914-1163-40e6-978a-ecf744667cbb admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:39.345884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47944dc2-9da0-4abd-badc-e60ed0d8bc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:39.346724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944dc2-9da0-4abd-badc-e60ed0d8bc91 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:39.347010 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944dc2-9da0-4abd-badc-e60ed0d8bc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:39.347289 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944dc2-9da0-4abd-badc-e60ed0d8bc91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:39.348051 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47944dc2-9da0-4abd-badc-e60ed0d8bc91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:39.349724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47944dc2-9da0-4abd-badc-e60ed0d8bc91 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:39.359718 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1111/2252] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:39.363889 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f6005-def6-4968-81a2-d6327c150ffd admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:39.376352 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f6005-def6-4968-81a2-d6327c150ffd admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:39.382423 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43034107-22f7-4b72-950b-daa1dbdd6896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:39.383376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43034107-22f7-4b72-950b-daa1dbdd6896 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:39.383658 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43034107-22f7-4b72-950b-daa1dbdd6896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:39.383932 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43034107-22f7-4b72-950b-daa1dbdd6896 admin admin] PATH_INFO: `/v3/projects/936f4081e609406b8d5214fe82223a0c/users/37881f4e58254ba4840d4ea1f9a5b163/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:39.385948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43034107-22f7-4b72-950b-daa1dbdd6896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=936f4081e609406b8d5214fe82223a0c, user_id=37881f4e58254ba4840d4ea1f9a5b163, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:39.388680 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43034107-22f7-4b72-950b-daa1dbdd6896 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:39.399175 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1142/2253] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:39 2022] PUT /identity/v3/projects/936f4081e609406b8d5214fe82223a0c/users/37881f4e58254ba4840d4ea1f9a5b163/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:39.414280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334217c4-337e-4504-8e17-da4b3ffebaf6 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:39.414280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334217c4-337e-4504-8e17-da4b3ffebaf6 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:39.414280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334217c4-337e-4504-8e17-da4b3ffebaf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:39.438455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-334217c4-337e-4504-8e17-da4b3ffebaf6 None None] MFA Rules not processed for user `37881f4e58254ba4840d4ea1f9a5b163`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:39.439686 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-334217c4-337e-4504-8e17-da4b3ffebaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:39.507608 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1112/2254] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:47:39.518580 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43034107-22f7-4b72-950b-daa1dbdd6896 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:39.520324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-43034107-22f7-4b72-950b-daa1dbdd6896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:39.579905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43034107-22f7-4b72-950b-daa1dbdd6896 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:39.583838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d665f-185a-4788-adc2-386dee64bc82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:39.584713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d665f-185a-4788-adc2-386dee64bc82 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:39.585128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d665f-185a-4788-adc2-386dee64bc82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:39.585412 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d665f-185a-4788-adc2-386dee64bc82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:39.594760 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3d665f-185a-4788-adc2-386dee64bc82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:39.596187 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3d665f-185a-4788-adc2-386dee64bc82 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:39.606566 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1143/2255] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:40.793088 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334217c4-337e-4504-8e17-da4b3ffebaf6 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:40.861908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:40.878997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334217c4-337e-4504-8e17-da4b3ffebaf6 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:40.884860 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbba180-b101-4540-8c28-f4e6b02a8ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:40.885788 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbba180-b101-4540-8c28-f4e6b02a8ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:40.886108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbba180-b101-4540-8c28-f4e6b02a8ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:40.886372 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbba180-b101-4540-8c28-f4e6b02a8ce8 admin admin] PATH_INFO: `/v3/users/c81ae9cec9d64bc993df2bc89374158f` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:40.887813 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbba180-b101-4540-8c28-f4e6b02a8ce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c81ae9cec9d64bc993df2bc89374158f)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:40.889935 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbba180-b101-4540-8c28-f4e6b02a8ce8 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:41.069115 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-1dbba180-b101-4540-8c28-f4e6b02a8ce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c81ae9cec9d64bc993df2bc89374158f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:41.081194 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-1dbba180-b101-4540-8c28-f4e6b02a8ce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c81ae9cec9d64bc993df2bc89374158f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:41.140722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-1dbba180-b101-4540-8c28-f4e6b02a8ce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c81ae9cec9d64bc993df2bc89374158f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:41.141348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-1dbba180-b101-4540-8c28-f4e6b02a8ce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c81ae9cec9d64bc993df2bc89374158f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:41.142046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-1dbba180-b101-4540-8c28-f4e6b02a8ce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c81ae9cec9d64bc993df2bc89374158f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:41.156077 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1113/2256] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:40 2022] DELETE /identity/v3/users/c81ae9cec9d64bc993df2bc89374158f => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:41.160799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d2bcc-465b-46bb-964b-1c368bc3fc38 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:41.161300 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d2bcc-465b-46bb-964b-1c368bc3fc38 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:41.161409 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d2bcc-465b-46bb-964b-1c368bc3fc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:41.180441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-809d2bcc-465b-46bb-964b-1c368bc3fc38 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:41.181615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-809d2bcc-465b-46bb-964b-1c368bc3fc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:41.245803 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1144/2257] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:47:41.258356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbba180-b101-4540-8c28-f4e6b02a8ce8 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:41.260301 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-1dbba180-b101-4540-8c28-f4e6b02a8ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:41.324079 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbba180-b101-4540-8c28-f4e6b02a8ce8 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:41.327788 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483cd165-666d-47bd-8b27-4edc727b5f88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:41.328433 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cd165-666d-47bd-8b27-4edc727b5f88 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:41.328753 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cd165-666d-47bd-8b27-4edc727b5f88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:41.329037 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cd165-666d-47bd-8b27-4edc727b5f88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:41.339102 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483cd165-666d-47bd-8b27-4edc727b5f88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:41.340541 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483cd165-666d-47bd-8b27-4edc727b5f88 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:41.350734 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1114/2258] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:41.423452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d2bcc-465b-46bb-964b-1c368bc3fc38 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:41.424940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-809d2bcc-465b-46bb-964b-1c368bc3fc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:41.437944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d2bcc-465b-46bb-964b-1c368bc3fc38 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:41.442523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c0f269-97bf-447a-81ba-349a61c2b0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:41.443390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0f269-97bf-447a-81ba-349a61c2b0a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:41.443734 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0f269-97bf-447a-81ba-349a61c2b0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:41.444040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0f269-97bf-447a-81ba-349a61c2b0a2 admin admin] PATH_INFO: `/v3/users/ad5b8ef100d441c6a978d2b5f0faef71` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:41.445289 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c0f269-97bf-447a-81ba-349a61c2b0a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad5b8ef100d441c6a978d2b5f0faef71)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:41.450083 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c0f269-97bf-447a-81ba-349a61c2b0a2 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:41.586380 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-84c0f269-97bf-447a-81ba-349a61c2b0a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad5b8ef100d441c6a978d2b5f0faef71'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:41.587734 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:41.602861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-84c0f269-97bf-447a-81ba-349a61c2b0a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad5b8ef100d441c6a978d2b5f0faef71'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:41.603721 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-84c0f269-97bf-447a-81ba-349a61c2b0a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad5b8ef100d441c6a978d2b5f0faef71'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:41.604703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-84c0f269-97bf-447a-81ba-349a61c2b0a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad5b8ef100d441c6a978d2b5f0faef71'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:41.613455 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-84c0f269-97bf-447a-81ba-349a61c2b0a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad5b8ef100d441c6a978d2b5f0faef71'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:41.690345 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1145/2259] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:41 2022] DELETE /identity/v3/users/ad5b8ef100d441c6a978d2b5f0faef71 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:41.695321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3ec86-47b8-4fcf-8d7b-1e6d1c2fd867 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:41.695907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3ec86-47b8-4fcf-8d7b-1e6d1c2fd867 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:41.696172 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3ec86-47b8-4fcf-8d7b-1e6d1c2fd867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:41.717480 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-33d3ec86-47b8-4fcf-8d7b-1e6d1c2fd867 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:41.719469 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-33d3ec86-47b8-4fcf-8d7b-1e6d1c2fd867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:41.756943 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c0f269-97bf-447a-81ba-349a61c2b0a2 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:41.758683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-84c0f269-97bf-447a-81ba-349a61c2b0a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:41.792061 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1115/2260] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:41.802139 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3ec86-47b8-4fcf-8d7b-1e6d1c2fd867 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:41.803901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-33d3ec86-47b8-4fcf-8d7b-1e6d1c2fd867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:41.818403 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:41.832379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c0f269-97bf-447a-81ba-349a61c2b0a2 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:41.838547 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf45715-1faf-44e4-9c72-5f38761382eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:41.839464 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf45715-1faf-44e4-9c72-5f38761382eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:41.839755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf45715-1faf-44e4-9c72-5f38761382eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:41.840036 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf45715-1faf-44e4-9c72-5f38761382eb admin admin] PATH_INFO: `/v3/projects/5c5a84e8906e4fa0b7175ead89168d4a` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:41.841264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf45715-1faf-44e4-9c72-5f38761382eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c5a84e8906e4fa0b7175ead89168d4a)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:41.843296 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf45715-1faf-44e4-9c72-5f38761382eb admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:41.869443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3ec86-47b8-4fcf-8d7b-1e6d1c2fd867 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:41.874730 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641857cd-eef2-44f1-94ce-3df5c51f182b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:41.875384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641857cd-eef2-44f1-94ce-3df5c51f182b service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:41.875671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641857cd-eef2-44f1-94ce-3df5c51f182b service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:41.875948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641857cd-eef2-44f1-94ce-3df5c51f182b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:41.886200 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641857cd-eef2-44f1-94ce-3df5c51f182b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:41.887959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641857cd-eef2-44f1-94ce-3df5c51f182b service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:41.898211 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1116/2261] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:41.985369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-9bf45715-1faf-44e4-9c72-5f38761382eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c5a84e8906e4fa0b7175ead89168d4a', 'request_id': 'req-9bf45715-1faf-44e4-9c72-5f38761382eb'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:41.993969 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-9bf45715-1faf-44e4-9c72-5f38761382eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c5a84e8906e4fa0b7175ead89168d4a', 'request_id': 'req-9bf45715-1faf-44e4-9c72-5f38761382eb'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:42.017291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-9bf45715-1faf-44e4-9c72-5f38761382eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c5a84e8906e4fa0b7175ead89168d4a', 'request_id': 'req-9bf45715-1faf-44e4-9c72-5f38761382eb'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:42.103314 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-9bf45715-1faf-44e4-9c72-5f38761382eb admin admin] The token cache is being invalidate because project 5c5a84e8906e4fa0b7175ead89168d4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:42.103686 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-9bf45715-1faf-44e4-9c72-5f38761382eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:42.104386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-9bf45715-1faf-44e4-9c72-5f38761382eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:42.106363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1146/2262] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:41 2022] DELETE /identity/v3/projects/5c5a84e8906e4fa0b7175ead89168d4a => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:42.109987 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641857cd-eef2-44f1-94ce-3df5c51f182b service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:42.124829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641857cd-eef2-44f1-94ce-3df5c51f182b service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:42.188548 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a3dc3-b998-4bc3-89ac-0466b3db74c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:42.189613 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a3dc3-b998-4bc3-89ac-0466b3db74c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:42.189993 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a3dc3-b998-4bc3-89ac-0466b3db74c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:42.190496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a3dc3-b998-4bc3-89ac-0466b3db74c6 admin admin] PATH_INFO: `/v3/users/37881f4e58254ba4840d4ea1f9a5b163` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:42.191800 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a3dc3-b998-4bc3-89ac-0466b3db74c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37881f4e58254ba4840d4ea1f9a5b163)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:42.194699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a3dc3-b998-4bc3-89ac-0466b3db74c6 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:42.354053 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf45715-1faf-44e4-9c72-5f38761382eb admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:42.356902 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-9bf45715-1faf-44e4-9c72-5f38761382eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:42.375760 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf45715-1faf-44e4-9c72-5f38761382eb admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:42.388124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e0788-0ea1-4b14-9dc7-cfe41353cd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:42.389397 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e0788-0ea1-4b14-9dc7-cfe41353cd6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:42.389890 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e0788-0ea1-4b14-9dc7-cfe41353cd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:42.390296 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e0788-0ea1-4b14-9dc7-cfe41353cd6d admin admin] PATH_INFO: `/v3/projects/566af4a530e444438e5353a805501741` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:42.391695 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e0788-0ea1-4b14-9dc7-cfe41353cd6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=566af4a530e444438e5353a805501741)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:42.393843 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e0788-0ea1-4b14-9dc7-cfe41353cd6d admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:42.397191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ed5a3dc3-b998-4bc3-89ac-0466b3db74c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37881f4e58254ba4840d4ea1f9a5b163'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:42.417032 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ed5a3dc3-b998-4bc3-89ac-0466b3db74c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37881f4e58254ba4840d4ea1f9a5b163'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:42.467327 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ed5a3dc3-b998-4bc3-89ac-0466b3db74c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37881f4e58254ba4840d4ea1f9a5b163'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:42.468477 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ed5a3dc3-b998-4bc3-89ac-0466b3db74c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37881f4e58254ba4840d4ea1f9a5b163'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:42.469158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ed5a3dc3-b998-4bc3-89ac-0466b3db74c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37881f4e58254ba4840d4ea1f9a5b163'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:42.484452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1117/2263] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:42 2022] DELETE /identity/v3/users/37881f4e58254ba4840d4ea1f9a5b163 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:42.488333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cb4e0788-0ea1-4b14-9dc7-cfe41353cd6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '566af4a530e444438e5353a805501741', 'request_id': 'req-cb4e0788-0ea1-4b14-9dc7-cfe41353cd6d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:42.489640 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:42.497076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cb4e0788-0ea1-4b14-9dc7-cfe41353cd6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '566af4a530e444438e5353a805501741', 'request_id': 'req-cb4e0788-0ea1-4b14-9dc7-cfe41353cd6d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:42.503438 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cb4e0788-0ea1-4b14-9dc7-cfe41353cd6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '566af4a530e444438e5353a805501741', 'request_id': 'req-cb4e0788-0ea1-4b14-9dc7-cfe41353cd6d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:42.552323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cb4e0788-0ea1-4b14-9dc7-cfe41353cd6d admin admin] The token cache is being invalidate because project 566af4a530e444438e5353a805501741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:42.552775 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cb4e0788-0ea1-4b14-9dc7-cfe41353cd6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:42.554637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cb4e0788-0ea1-4b14-9dc7-cfe41353cd6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:42.556347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1147/2264] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:42 2022] DELETE /identity/v3/projects/566af4a530e444438e5353a805501741 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:42.561696 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b684e4-4259-4641-9c8c-71ca9100ba54 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:42.561845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b684e4-4259-4641-9c8c-71ca9100ba54 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:42.562292 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b684e4-4259-4641-9c8c-71ca9100ba54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:42.583025 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-70b684e4-4259-4641-9c8c-71ca9100ba54 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:42.584751 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-70b684e4-4259-4641-9c8c-71ca9100ba54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:42.647468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1118/2265] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:42.655363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ccb1c-99b0-4b28-b3c8-ad9aeca99dc0 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:42.655755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ccb1c-99b0-4b28-b3c8-ad9aeca99dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:42.656123 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ccb1c-99b0-4b28-b3c8-ad9aeca99dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:42.681130 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-de5ccb1c-99b0-4b28-b3c8-ad9aeca99dc0 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:42.682840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-de5ccb1c-99b0-4b28-b3c8-ad9aeca99dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:42.694572 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1148/2266] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:42.699630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b684e4-4259-4641-9c8c-71ca9100ba54 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:42.715051 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b684e4-4259-4641-9c8c-71ca9100ba54 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:42.720077 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e195731-cbd3-4d27-9595-4e199f7924b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:42.720936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e195731-cbd3-4d27-9595-4e199f7924b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:42.721240 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e195731-cbd3-4d27-9595-4e199f7924b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:42.721510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e195731-cbd3-4d27-9595-4e199f7924b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:42.722298 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e195731-cbd3-4d27-9595-4e199f7924b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:47:42.722662 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e195731-cbd3-4d27-9595-4e199f7924b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:42.733251 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e195731-cbd3-4d27-9595-4e199f7924b6 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:42.744669 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:42.749837 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1119/2267] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:47:42.754531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ccb1c-99b0-4b28-b3c8-ad9aeca99dc0 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:42.768555 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ccb1c-99b0-4b28-b3c8-ad9aeca99dc0 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:42.790123 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d92e31-0fe8-46ed-9f9d-5304c106911a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:42.790840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d92e31-0fe8-46ed-9f9d-5304c106911a admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:42.791130 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d92e31-0fe8-46ed-9f9d-5304c106911a admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:42.791579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d92e31-0fe8-46ed-9f9d-5304c106911a admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:42.792392 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d92e31-0fe8-46ed-9f9d-5304c106911a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:42.794556 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d92e31-0fe8-46ed-9f9d-5304c106911a admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:42.902704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1149/2268] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:42 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:47:42.906512 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e195731-cbd3-4d27-9595-4e199f7924b6 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:42.918025 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e195731-cbd3-4d27-9595-4e199f7924b6 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:42.971791 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d92e31-0fe8-46ed-9f9d-5304c106911a admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:42.973883 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-b2d92e31-0fe8-46ed-9f9d-5304c106911a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:42.988064 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b0388-4f8c-4de8-9a45-02a0d82a3e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:42.988954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b0388-4f8c-4de8-9a45-02a0d82a3e54 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:42.989344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b0388-4f8c-4de8-9a45-02a0d82a3e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:42.989631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b0388-4f8c-4de8-9a45-02a0d82a3e54 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:42.990383 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b0388-4f8c-4de8-9a45-02a0d82a3e54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:42.992630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b0388-4f8c-4de8-9a45-02a0d82a3e54 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:43.072902 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1120/2269] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:47:42 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:47:43.075549 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b0388-4f8c-4de8-9a45-02a0d82a3e54 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:43.103024 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b0388-4f8c-4de8-9a45-02a0d82a3e54 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:43.105855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d743f-aee4-4e7a-bb24-9d958ab70039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:43.106565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d743f-aee4-4e7a-bb24-9d958ab70039 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:43.106848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d743f-aee4-4e7a-bb24-9d958ab70039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:43.107402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d743f-aee4-4e7a-bb24-9d958ab70039 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:43.108270 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d743f-aee4-4e7a-bb24-9d958ab70039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:43.110793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d743f-aee4-4e7a-bb24-9d958ab70039 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:43.141166 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1121/2270] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:43.145540 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d743f-aee4-4e7a-bb24-9d958ab70039 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:43.158746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d92e31-0fe8-46ed-9f9d-5304c106911a admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:43.167174 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d743f-aee4-4e7a-bb24-9d958ab70039 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:43.171975 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd94d4a-6aa3-4075-a315-28a0f58a82fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:43.173425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd94d4a-6aa3-4075-a315-28a0f58a82fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:43.174844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd94d4a-6aa3-4075-a315-28a0f58a82fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:43.175334 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd94d4a-6aa3-4075-a315-28a0f58a82fb admin admin] PATH_INFO: `/v3/projects/936f4081e609406b8d5214fe82223a0c` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:43.178235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd94d4a-6aa3-4075-a315-28a0f58a82fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=936f4081e609406b8d5214fe82223a0c)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:43.183592 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd94d4a-6aa3-4075-a315-28a0f58a82fb admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:43.184411 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891b1bd-8931-4c1b-8ec7-64334fb0b68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:43.185778 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891b1bd-8931-4c1b-8ec7-64334fb0b68e admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:43.186312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891b1bd-8931-4c1b-8ec7-64334fb0b68e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:43.187141 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891b1bd-8931-4c1b-8ec7-64334fb0b68e admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:43.188213 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891b1bd-8931-4c1b-8ec7-64334fb0b68e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:43.190165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891b1bd-8931-4c1b-8ec7-64334fb0b68e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:43.210823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1122/2271] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:43.215316 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891b1bd-8931-4c1b-8ec7-64334fb0b68e admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:43.234939 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891b1bd-8931-4c1b-8ec7-64334fb0b68e admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:43.245025 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a9fd3-a8d2-4d88-ad3e-030d80119acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:43.248961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a9fd3-a8d2-4d88-ad3e-030d80119acb admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:43.249675 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a9fd3-a8d2-4d88-ad3e-030d80119acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:43.250190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a9fd3-a8d2-4d88-ad3e-030d80119acb admin admin] PATH_INFO: `/v3/projects/a3ed29eff54c4d0299306af9b3cfeaaf/users/5505e25902e64fa898249376d769fd2c/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:43.274695 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a9fd3-a8d2-4d88-ad3e-030d80119acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3ed29eff54c4d0299306af9b3cfeaaf, user_id=5505e25902e64fa898249376d769fd2c, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:43.277774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a9fd3-a8d2-4d88-ad3e-030d80119acb admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:43.383990 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1123/2272] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:43 2022] PUT /identity/v3/projects/a3ed29eff54c4d0299306af9b3cfeaaf/users/5505e25902e64fa898249376d769fd2c/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:43.396349 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-efd94d4a-6aa3-4075-a315-28a0f58a82fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '936f4081e609406b8d5214fe82223a0c', 'request_id': 'req-efd94d4a-6aa3-4075-a315-28a0f58a82fb'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:43.401687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40166abe-9983-4ea7-b449-594326184b8c None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:43.402190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40166abe-9983-4ea7-b449-594326184b8c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:43.402408 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40166abe-9983-4ea7-b449-594326184b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:43.414965 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-efd94d4a-6aa3-4075-a315-28a0f58a82fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '936f4081e609406b8d5214fe82223a0c', 'request_id': 'req-efd94d4a-6aa3-4075-a315-28a0f58a82fb'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:43.425840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-efd94d4a-6aa3-4075-a315-28a0f58a82fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '936f4081e609406b8d5214fe82223a0c', 'request_id': 'req-efd94d4a-6aa3-4075-a315-28a0f58a82fb'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:43.439721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-40166abe-9983-4ea7-b449-594326184b8c None None] MFA Rules not processed for user `5505e25902e64fa898249376d769fd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:43.441344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-40166abe-9983-4ea7-b449-594326184b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:43.531731 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1124/2273] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:43.565347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-efd94d4a-6aa3-4075-a315-28a0f58a82fb admin admin] The token cache is being invalidate because project 936f4081e609406b8d5214fe82223a0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:43.565647 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-efd94d4a-6aa3-4075-a315-28a0f58a82fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:43.567446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-efd94d4a-6aa3-4075-a315-28a0f58a82fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:43.569993 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1150/2274] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:42 2022] DELETE /identity/v3/projects/936f4081e609406b8d5214fe82223a0c => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:43.578926 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe94f1-4ad6-4dd2-b729-d047cca71035 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:43.578926 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe94f1-4ad6-4dd2-b729-d047cca71035 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:43.578926 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe94f1-4ad6-4dd2-b729-d047cca71035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:43.605863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-fabe94f1-4ad6-4dd2-b729-d047cca71035 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:43.607993 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-fabe94f1-4ad6-4dd2-b729-d047cca71035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:43.695677 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd94d4a-6aa3-4075-a315-28a0f58a82fb admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:43.697659 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-efd94d4a-6aa3-4075-a315-28a0f58a82fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:43.700344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:43.711401 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1125/2275] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:43.713872 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174314a7-d73e-45c3-8e2f-f96538a51752 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:43.714130 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174314a7-d73e-45c3-8e2f-f96538a51752 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:43.714418 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174314a7-d73e-45c3-8e2f-f96538a51752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:43.741498 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-174314a7-d73e-45c3-8e2f-f96538a51752 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:43.743044 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-174314a7-d73e-45c3-8e2f-f96538a51752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:43.751402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1126/2276] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:43.755743 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174314a7-d73e-45c3-8e2f-f96538a51752 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:43.767504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174314a7-d73e-45c3-8e2f-f96538a51752 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:43.772551 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eab844-d260-4fd9-8559-ef677221013a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:43.773255 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eab844-d260-4fd9-8559-ef677221013a admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:43.773594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eab844-d260-4fd9-8559-ef677221013a admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:43.773904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eab844-d260-4fd9-8559-ef677221013a admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:43.774788 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eab844-d260-4fd9-8559-ef677221013a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:47:43.775103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eab844-d260-4fd9-8559-ef677221013a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:43.776797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eab844-d260-4fd9-8559-ef677221013a admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:43.779040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:43.793343 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1127/2277] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:47:43.802002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eab844-d260-4fd9-8559-ef677221013a admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:43.802087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd94d4a-6aa3-4075-a315-28a0f58a82fb admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:43.805547 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13bdc4-a4af-45be-9079-ce1f0382b4ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:43.806215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13bdc4-a4af-45be-9079-ce1f0382b4ec service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:43.806564 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13bdc4-a4af-45be-9079-ce1f0382b4ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:43.806854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13bdc4-a4af-45be-9079-ce1f0382b4ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:43.809058 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-7b13bdc4-a4af-45be-9079-ce1f0382b4ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:43.826816 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eab844-d260-4fd9-8559-ef677221013a admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:43.832658 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13bdc4-a4af-45be-9079-ce1f0382b4ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:43.840738 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13bdc4-a4af-45be-9079-ce1f0382b4ec service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:43.841900 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c8deb-f74a-4998-bc35-8a2432e4ced4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:43.842639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c8deb-f74a-4998-bc35-8a2432e4ced4 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:43.850451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c8deb-f74a-4998-bc35-8a2432e4ced4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:43.850451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c8deb-f74a-4998-bc35-8a2432e4ced4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:43.850451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c8deb-f74a-4998-bc35-8a2432e4ced4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:43.850451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c8deb-f74a-4998-bc35-8a2432e4ced4 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:43.864673 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1151/2278] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:43.974423 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1128/2279] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:43 2022] POST /identity/v3/projects => generated 383 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:43.979063 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13bdc4-a4af-45be-9079-ce1f0382b4ec service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:43.992477 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13bdc4-a4af-45be-9079-ce1f0382b4ec service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:44.081996 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd62885-d86b-4604-b3f4-80046be72f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:44.082679 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd62885-d86b-4604-b3f4-80046be72f70 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:44.082971 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd62885-d86b-4604-b3f4-80046be72f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:44.083254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd62885-d86b-4604-b3f4-80046be72f70 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:44.084121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd62885-d86b-4604-b3f4-80046be72f70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:44.086272 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd62885-d86b-4604-b3f4-80046be72f70 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:44.204344 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1152/2280] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:47:43 2022] POST /identity/v3/users => generated 292 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:44.208475 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c8deb-f74a-4998-bc35-8a2432e4ced4 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:44.218771 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c8deb-f74a-4998-bc35-8a2432e4ced4 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:44.223697 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f77f5-2bcf-4553-95a7-b1c96164f05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:44.225550 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f77f5-2bcf-4553-95a7-b1c96164f05f admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:44.225550 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f77f5-2bcf-4553-95a7-b1c96164f05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:44.225550 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f77f5-2bcf-4553-95a7-b1c96164f05f admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:44.226257 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f77f5-2bcf-4553-95a7-b1c96164f05f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:44.227852 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f77f5-2bcf-4553-95a7-b1c96164f05f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:44.238147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1129/2281] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:44.248496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd62885-d86b-4604-b3f4-80046be72f70 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:44.253287 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd62885-d86b-4604-b3f4-80046be72f70 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:44.258236 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d261c0-0e6c-44f6-97cb-dbedee45d44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:44.258897 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d261c0-0e6c-44f6-97cb-dbedee45d44e admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:44.259247 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d261c0-0e6c-44f6-97cb-dbedee45d44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:44.259512 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d261c0-0e6c-44f6-97cb-dbedee45d44e admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:44.260240 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d261c0-0e6c-44f6-97cb-dbedee45d44e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:44.262178 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d261c0-0e6c-44f6-97cb-dbedee45d44e admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:44.272362 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1153/2282] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:44.276326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f77f5-2bcf-4553-95a7-b1c96164f05f admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:44.286667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f77f5-2bcf-4553-95a7-b1c96164f05f admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:44.292975 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2e246-4c05-4759-9eff-bdcdef6bfb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:44.294011 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2e246-4c05-4759-9eff-bdcdef6bfb95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:44.294347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2e246-4c05-4759-9eff-bdcdef6bfb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:44.294576 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2e246-4c05-4759-9eff-bdcdef6bfb95 admin admin] PATH_INFO: `/v3/projects/ebac7a1ed527462fbb0ab91d84bf5771/users/b74f6c9c99824e22a599d5cbe5eef46e/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:44.311973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2e246-4c05-4759-9eff-bdcdef6bfb95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebac7a1ed527462fbb0ab91d84bf5771, user_id=b74f6c9c99824e22a599d5cbe5eef46e, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:44.314925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2e246-4c05-4759-9eff-bdcdef6bfb95 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:44.399958 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1130/2283] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:44 2022] PUT /identity/v3/projects/ebac7a1ed527462fbb0ab91d84bf5771/users/b74f6c9c99824e22a599d5cbe5eef46e/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:44.410793 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0f2d7-593c-430b-90fa-944e16677734 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:44.413676 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0f2d7-593c-430b-90fa-944e16677734 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:44.414180 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0f2d7-593c-430b-90fa-944e16677734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:44.448035 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-7ce0f2d7-593c-430b-90fa-944e16677734 None None] MFA Rules not processed for user `b74f6c9c99824e22a599d5cbe5eef46e`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:44.449219 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-7ce0f2d7-593c-430b-90fa-944e16677734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:44.520514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1154/2284] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:44.526340 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2e246-4c05-4759-9eff-bdcdef6bfb95 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:44.593155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2e246-4c05-4759-9eff-bdcdef6bfb95 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:44.601275 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06188e4e-1f58-476d-9564-e1c499f8c3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:44.602593 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06188e4e-1f58-476d-9564-e1c499f8c3ee admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:44.602741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06188e4e-1f58-476d-9564-e1c499f8c3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:44.603428 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06188e4e-1f58-476d-9564-e1c499f8c3ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:44.604026 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06188e4e-1f58-476d-9564-e1c499f8c3ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:44.606134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06188e4e-1f58-476d-9564-e1c499f8c3ee admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:44.707720 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1131/2285] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:44 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:44.713022 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0f2d7-593c-430b-90fa-944e16677734 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:44.723880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0f2d7-593c-430b-90fa-944e16677734 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:44.787489 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:44.791330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950a779-c16c-4d69-8dbe-fd31babf225c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:44.792021 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950a779-c16c-4d69-8dbe-fd31babf225c admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:44.792314 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950a779-c16c-4d69-8dbe-fd31babf225c admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:44.792655 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950a779-c16c-4d69-8dbe-fd31babf225c admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:44.793366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950a779-c16c-4d69-8dbe-fd31babf225c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:44.795301 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950a779-c16c-4d69-8dbe-fd31babf225c admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:44.874609 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1155/2286] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:47:44 2022] POST /identity/v3/users => generated 299 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:44.878241 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06188e4e-1f58-476d-9564-e1c499f8c3ee admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:44.884909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:44.890668 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06188e4e-1f58-476d-9564-e1c499f8c3ee admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:44.900964 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715053a7-9459-4f4a-8976-38bebe52a108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:44.904926 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715053a7-9459-4f4a-8976-38bebe52a108 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:44.904926 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715053a7-9459-4f4a-8976-38bebe52a108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:44.904926 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715053a7-9459-4f4a-8976-38bebe52a108 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:44.904926 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715053a7-9459-4f4a-8976-38bebe52a108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:44.905609 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715053a7-9459-4f4a-8976-38bebe52a108 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:44.919744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1132/2287] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:44.920967 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950a779-c16c-4d69-8dbe-fd31babf225c admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:44.932022 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950a779-c16c-4d69-8dbe-fd31babf225c admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:44.937106 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b8393-5a33-4925-8a01-2b8d87d312f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:44.938005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b8393-5a33-4925-8a01-2b8d87d312f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:44.938407 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b8393-5a33-4925-8a01-2b8d87d312f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:44.938708 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b8393-5a33-4925-8a01-2b8d87d312f6 admin admin] PATH_INFO: `/v3/projects/1c3c2031ab43423bb8467b4005ba401c/users/c6b4ce1be2ed45438fce9d7ed798d64d/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:44.955878 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b8393-5a33-4925-8a01-2b8d87d312f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c3c2031ab43423bb8467b4005ba401c, user_id=c6b4ce1be2ed45438fce9d7ed798d64d, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:44.959630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b8393-5a33-4925-8a01-2b8d87d312f6 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:45.042771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1156/2288] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:44 2022] PUT /identity/v3/projects/1c3c2031ab43423bb8467b4005ba401c/users/c6b4ce1be2ed45438fce9d7ed798d64d/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:45.045706 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715053a7-9459-4f4a-8976-38bebe52a108 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:45.057023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715053a7-9459-4f4a-8976-38bebe52a108 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:45.112269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc3d75-42f9-410c-b71e-d2b14b71cb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:45.112991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc3d75-42f9-410c-b71e-d2b14b71cb29 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:45.113281 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc3d75-42f9-410c-b71e-d2b14b71cb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:45.113566 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc3d75-42f9-410c-b71e-d2b14b71cb29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:45.114292 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc3d75-42f9-410c-b71e-d2b14b71cb29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:45.116812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc3d75-42f9-410c-b71e-d2b14b71cb29 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:45.126869 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1133/2289] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:45.131710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b8393-5a33-4925-8a01-2b8d87d312f6 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:45.142215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b8393-5a33-4925-8a01-2b8d87d312f6 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:45.147232 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af79cc-c69b-47c0-92bb-331bc3f77acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:45.148124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af79cc-c69b-47c0-92bb-331bc3f77acb admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:45.148409 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af79cc-c69b-47c0-92bb-331bc3f77acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:45.148745 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af79cc-c69b-47c0-92bb-331bc3f77acb admin admin] PATH_INFO: `/v3/projects/1c3c2031ab43423bb8467b4005ba401c/users/c6b4ce1be2ed45438fce9d7ed798d64d/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:45.150884 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af79cc-c69b-47c0-92bb-331bc3f77acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c3c2031ab43423bb8467b4005ba401c, user_id=c6b4ce1be2ed45438fce9d7ed798d64d, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:45.153603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af79cc-c69b-47c0-92bb-331bc3f77acb admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:45.164648 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1157/2290] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:45 2022] PUT /identity/v3/projects/1c3c2031ab43423bb8467b4005ba401c/users/c6b4ce1be2ed45438fce9d7ed798d64d/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:45.175681 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65598b3-7d75-4358-b15a-02fa522e289d None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:45.175981 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65598b3-7d75-4358-b15a-02fa522e289d None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:45.176268 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65598b3-7d75-4358-b15a-02fa522e289d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:45.204300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-f65598b3-7d75-4358-b15a-02fa522e289d None None] MFA Rules not processed for user `c6b4ce1be2ed45438fce9d7ed798d64d`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:45.207690 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-f65598b3-7d75-4358-b15a-02fa522e289d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:45.273878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1134/2291] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:45 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:45.284728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af79cc-c69b-47c0-92bb-331bc3f77acb admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:45.352488 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af79cc-c69b-47c0-92bb-331bc3f77acb admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:45.354786 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080bd165-ddf0-48b0-b896-3284bf130bb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:45.355434 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bd165-ddf0-48b0-b896-3284bf130bb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:45.358430 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bd165-ddf0-48b0-b896-3284bf130bb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:45.358430 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bd165-ddf0-48b0-b896-3284bf130bb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:45.414482 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080bd165-ddf0-48b0-b896-3284bf130bb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:45.415919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080bd165-ddf0-48b0-b896-3284bf130bb3 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:45.424807 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1158/2292] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:47:47.376433 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e2227-6d44-4b80-bc03-50729dec6348 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:47.378234 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e2227-6d44-4b80-bc03-50729dec6348 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:47.378234 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e2227-6d44-4b80-bc03-50729dec6348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:47.394930 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-252e2227-6d44-4b80-bc03-50729dec6348 None None] MFA Rules not processed for user `675d117397da452faa5221af0906fb12`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:47.396151 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-252e2227-6d44-4b80-bc03-50729dec6348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:47.451523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1135/2293] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:47:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:55.549924 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=73,keystone:INSERT=5 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:57.444743 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=92 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:00.722293 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080bd165-ddf0-48b0-b896-3284bf130bb3 service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:00.724290 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-080bd165-ddf0-48b0-b896-3284bf130bb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:00.748998 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080bd165-ddf0-48b0-b896-3284bf130bb3 service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:00.752908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c42777-adb1-4cb0-9330-d75efbe2f932 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:00.753630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c42777-adb1-4cb0-9330-d75efbe2f932 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:00.753944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c42777-adb1-4cb0-9330-d75efbe2f932 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:00.754234 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c42777-adb1-4cb0-9330-d75efbe2f932 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:00.763978 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c42777-adb1-4cb0-9330-d75efbe2f932 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:00.765466 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c42777-adb1-4cb0-9330-d75efbe2f932 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:00.776537 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1159/2294] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:48:02.071259 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e2227-6d44-4b80-bc03-50729dec6348 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:02.073883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-252e2227-6d44-4b80-bc03-50729dec6348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:02.154686 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e2227-6d44-4b80-bc03-50729dec6348 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:02.159438 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b36f2-709d-47eb-858f-db1faf63a841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:02.160411 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b36f2-709d-47eb-858f-db1faf63a841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:02.160733 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b36f2-709d-47eb-858f-db1faf63a841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:02.161027 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b36f2-709d-47eb-858f-db1faf63a841 admin admin] PATH_INFO: `/v3/users/5505e25902e64fa898249376d769fd2c` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:02.162603 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b36f2-709d-47eb-858f-db1faf63a841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5505e25902e64fa898249376d769fd2c)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:02.164658 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b36f2-709d-47eb-858f-db1faf63a841 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:02.378147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-cc5b36f2-709d-47eb-858f-db1faf63a841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5505e25902e64fa898249376d769fd2c'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:02.391575 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-cc5b36f2-709d-47eb-858f-db1faf63a841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5505e25902e64fa898249376d769fd2c'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:02.478214 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-cc5b36f2-709d-47eb-858f-db1faf63a841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5505e25902e64fa898249376d769fd2c'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:02.478972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-cc5b36f2-709d-47eb-858f-db1faf63a841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5505e25902e64fa898249376d769fd2c'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:02.479707 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-cc5b36f2-709d-47eb-858f-db1faf63a841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5505e25902e64fa898249376d769fd2c'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:02.494330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1136/2295] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:48:02 2022] DELETE /identity/v3/users/5505e25902e64fa898249376d769fd2c => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:02.498837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98140b85-ce27-4259-9717-bf43a39ea7e1 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:02.499291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98140b85-ce27-4259-9717-bf43a39ea7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:02.499455 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98140b85-ce27-4259-9717-bf43a39ea7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:02.518731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-98140b85-ce27-4259-9717-bf43a39ea7e1 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:02.519951 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-98140b85-ce27-4259-9717-bf43a39ea7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:02.593945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1160/2296] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:48:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:02.603561 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b36f2-709d-47eb-858f-db1faf63a841 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:02.605106 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-cc5b36f2-709d-47eb-858f-db1faf63a841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:02.672558 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b36f2-709d-47eb-858f-db1faf63a841 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:02.676240 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbeff8f-c60d-4323-aba1-566058f2a21d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:02.677529 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbeff8f-c60d-4323-aba1-566058f2a21d service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:02.677752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbeff8f-c60d-4323-aba1-566058f2a21d service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:02.680301 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbeff8f-c60d-4323-aba1-566058f2a21d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:02.694196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbeff8f-c60d-4323-aba1-566058f2a21d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:02.695474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbeff8f-c60d-4323-aba1-566058f2a21d service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:02.706210 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1137/2297] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:48:03.153880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98140b85-ce27-4259-9717-bf43a39ea7e1 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:03.155526 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-98140b85-ce27-4259-9717-bf43a39ea7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:03.175040 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98140b85-ce27-4259-9717-bf43a39ea7e1 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:03.182018 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6d9cb-b566-466f-80e7-bf4a555440b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:03.183068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6d9cb-b566-466f-80e7-bf4a555440b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:03.183482 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6d9cb-b566-466f-80e7-bf4a555440b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:03.183854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6d9cb-b566-466f-80e7-bf4a555440b4 admin admin] PATH_INFO: `/v3/projects/a3ed29eff54c4d0299306af9b3cfeaaf` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:03.193473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6d9cb-b566-466f-80e7-bf4a555440b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3ed29eff54c4d0299306af9b3cfeaaf)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:03.195502 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6d9cb-b566-466f-80e7-bf4a555440b4 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:03.365970 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6ff6d9cb-b566-466f-80e7-bf4a555440b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3ed29eff54c4d0299306af9b3cfeaaf', 'request_id': 'req-6ff6d9cb-b566-466f-80e7-bf4a555440b4'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:03.373645 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6ff6d9cb-b566-466f-80e7-bf4a555440b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3ed29eff54c4d0299306af9b3cfeaaf', 'request_id': 'req-6ff6d9cb-b566-466f-80e7-bf4a555440b4'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:03.379293 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6ff6d9cb-b566-466f-80e7-bf4a555440b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3ed29eff54c4d0299306af9b3cfeaaf', 'request_id': 'req-6ff6d9cb-b566-466f-80e7-bf4a555440b4'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:03.477100 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6ff6d9cb-b566-466f-80e7-bf4a555440b4 admin admin] The token cache is being invalidate because project a3ed29eff54c4d0299306af9b3cfeaaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:48:03.477393 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6ff6d9cb-b566-466f-80e7-bf4a555440b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:03.478029 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6ff6d9cb-b566-466f-80e7-bf4a555440b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:03.479590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1161/2298] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:48:03 2022] DELETE /identity/v3/projects/a3ed29eff54c4d0299306af9b3cfeaaf => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:03.484556 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2f09f-e615-413d-a4b4-1038d04a3486 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:03.485073 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2f09f-e615-413d-a4b4-1038d04a3486 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:03.485172 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2f09f-e615-413d-a4b4-1038d04a3486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:03.503997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-d8e2f09f-e615-413d-a4b4-1038d04a3486 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:03.505785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-d8e2f09f-e615-413d-a4b4-1038d04a3486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:03.523477 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:03.564324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1138/2299] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:03.573001 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c0f74-05af-4f95-811f-46ce4b714c7d None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:03.573495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c0f74-05af-4f95-811f-46ce4b714c7d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:03.573691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c0f74-05af-4f95-811f-46ce4b714c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:03.592412 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-018c0f74-05af-4f95-811f-46ce4b714c7d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:03.595685 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-018c0f74-05af-4f95-811f-46ce4b714c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:03.601807 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1162/2300] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:03.605892 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e2f09f-e615-413d-a4b4-1038d04a3486 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:03.615930 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e2f09f-e615-413d-a4b4-1038d04a3486 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:03.625286 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f7ac0-eb6c-4532-acb6-036ba6308848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:03.625286 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f7ac0-eb6c-4532-acb6-036ba6308848 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:03.625286 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f7ac0-eb6c-4532-acb6-036ba6308848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:03.625286 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f7ac0-eb6c-4532-acb6-036ba6308848 admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:03.625687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f7ac0-eb6c-4532-acb6-036ba6308848 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:48:03.626027 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f7ac0-eb6c-4532-acb6-036ba6308848 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:03.627710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f7ac0-eb6c-4532-acb6-036ba6308848 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:03.642893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1139/2301] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:48:03.647893 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c0f74-05af-4f95-811f-46ce4b714c7d None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:03.658268 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c0f74-05af-4f95-811f-46ce4b714c7d None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:03.663095 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5289f-4766-40aa-9d42-45b4d73b823c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:03.663785 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5289f-4766-40aa-9d42-45b4d73b823c admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:03.667728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5289f-4766-40aa-9d42-45b4d73b823c admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:03.667728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5289f-4766-40aa-9d42-45b4d73b823c admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:03.667728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a5289f-4766-40aa-9d42-45b4d73b823c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:03.667728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a5289f-4766-40aa-9d42-45b4d73b823c admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:03.758363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1163/2302] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:48:03 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:48:03.762455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f7ac0-eb6c-4532-acb6-036ba6308848 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:03.774038 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f7ac0-eb6c-4532-acb6-036ba6308848 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:03.845768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e824eba-2ab2-45eb-8461-4c16c14b9cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:03.846620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e824eba-2ab2-45eb-8461-4c16c14b9cc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:03.846911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e824eba-2ab2-45eb-8461-4c16c14b9cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:03.847189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e824eba-2ab2-45eb-8461-4c16c14b9cc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:03.847898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e824eba-2ab2-45eb-8461-4c16c14b9cc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:03.850125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e824eba-2ab2-45eb-8461-4c16c14b9cc4 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:03.957292 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1140/2303] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:48:03 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:48:03.961572 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5289f-4766-40aa-9d42-45b4d73b823c admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:03.971932 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5289f-4766-40aa-9d42-45b4d73b823c admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:03.983113 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61997dd3-382f-489f-8b11-c6f894988ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:03.983800 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61997dd3-382f-489f-8b11-c6f894988ada admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:03.984255 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61997dd3-382f-489f-8b11-c6f894988ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:03.984721 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61997dd3-382f-489f-8b11-c6f894988ada admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:03.985613 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61997dd3-382f-489f-8b11-c6f894988ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:03.987225 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61997dd3-382f-489f-8b11-c6f894988ada admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:04.011519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1164/2304] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:48:04.015474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e824eba-2ab2-45eb-8461-4c16c14b9cc4 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:04.027543 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e824eba-2ab2-45eb-8461-4c16c14b9cc4 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:04.033648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36d9bf-4ac3-4f1e-a2c1-fd2b90cd5744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:04.034500 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36d9bf-4ac3-4f1e-a2c1-fd2b90cd5744 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:04.034927 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36d9bf-4ac3-4f1e-a2c1-fd2b90cd5744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:04.036620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36d9bf-4ac3-4f1e-a2c1-fd2b90cd5744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:04.036620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36d9bf-4ac3-4f1e-a2c1-fd2b90cd5744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:04.038132 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36d9bf-4ac3-4f1e-a2c1-fd2b90cd5744 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:04.050613 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1141/2305] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:48:04.055339 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61997dd3-382f-489f-8b11-c6f894988ada admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:04.066954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61997dd3-382f-489f-8b11-c6f894988ada admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:04.073021 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff5269-c9c7-4ab9-a5ed-832a722ef4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:04.074034 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff5269-c9c7-4ab9-a5ed-832a722ef4de admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:04.074380 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff5269-c9c7-4ab9-a5ed-832a722ef4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:04.074756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff5269-c9c7-4ab9-a5ed-832a722ef4de admin admin] PATH_INFO: `/v3/projects/7b88b05dded7480d8f825e3c320fbc8d/users/3e47682645964d1797d13ddb62003bac/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:04.090272 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff5269-c9c7-4ab9-a5ed-832a722ef4de admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b88b05dded7480d8f825e3c320fbc8d, user_id=3e47682645964d1797d13ddb62003bac, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:04.092973 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff5269-c9c7-4ab9-a5ed-832a722ef4de admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:04.096001 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:04.209098 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1165/2306] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:48:04 2022] PUT /identity/v3/projects/7b88b05dded7480d8f825e3c320fbc8d/users/3e47682645964d1797d13ddb62003bac/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:04.219142 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a4cfe-5a7f-457d-8a9e-1d67943940d4 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:04.219436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a4cfe-5a7f-457d-8a9e-1d67943940d4 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:04.219719 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a4cfe-5a7f-457d-8a9e-1d67943940d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:04.247344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-ab2a4cfe-5a7f-457d-8a9e-1d67943940d4 None None] MFA Rules not processed for user `3e47682645964d1797d13ddb62003bac`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:04.248552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ab2a4cfe-5a7f-457d-8a9e-1d67943940d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:04.316760 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1142/2307] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:48:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:04.327587 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff5269-c9c7-4ab9-a5ed-832a722ef4de admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:04.329382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-5bff5269-c9c7-4ab9-a5ed-832a722ef4de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:04.388550 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff5269-c9c7-4ab9-a5ed-832a722ef4de admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:04.392393 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc5129-4ab4-4468-87eb-baf70d5d5220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:04.393064 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc5129-4ab4-4468-87eb-baf70d5d5220 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:04.393354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc5129-4ab4-4468-87eb-baf70d5d5220 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:04.393633 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc5129-4ab4-4468-87eb-baf70d5d5220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:04.403530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc5129-4ab4-4468-87eb-baf70d5d5220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:04.405708 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc5129-4ab4-4468-87eb-baf70d5d5220 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:04.418013 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1166/2308] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:48:07.742398 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06403569-2501-4ecb-9ebd-75a828aa6b66 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:07.743616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06403569-2501-4ecb-9ebd-75a828aa6b66 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:07.743921 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06403569-2501-4ecb-9ebd-75a828aa6b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:07.773577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:07.807275 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-06403569-2501-4ecb-9ebd-75a828aa6b66 None None] MFA Rules not processed for user `0a7c204a298c42f0bbc32fa9a6ce1f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:07.808813 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-06403569-2501-4ecb-9ebd-75a828aa6b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:07.882064 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1143/2309] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 22:48:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:07.928744 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc5129-4ab4-4468-87eb-baf70d5d5220 service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:07.931055 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-f5cc5129-4ab4-4468-87eb-baf70d5d5220 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:08.037825 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc5129-4ab4-4468-87eb-baf70d5d5220 service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:08.044129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7cb66-b079-4449-a1ea-13235515c7d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b321586cdb3d4d25904472aaa4adba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:08.044825 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7cb66-b079-4449-a1ea-13235515c7d2 service swift] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:08.045127 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7cb66-b079-4449-a1ea-13235515c7d2 service swift] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:08.045399 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7cb66-b079-4449-a1ea-13235515c7d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:08.054556 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7cb66-b079-4449-a1ea-13235515c7d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:08.059502 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7cb66-b079-4449-a1ea-13235515c7d2 service swift] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:08.066136 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1167/2310] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 22:48:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 22:48:12.411928 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06403569-2501-4ecb-9ebd-75a828aa6b66 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:12.414444 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-06403569-2501-4ecb-9ebd-75a828aa6b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:12.561918 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06403569-2501-4ecb-9ebd-75a828aa6b66 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:12.569545 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636a835-ace4-49e9-ba67-b1a9ec34a420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:12.570552 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636a835-ace4-49e9-ba67-b1a9ec34a420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:12.570838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636a835-ace4-49e9-ba67-b1a9ec34a420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:12.571172 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636a835-ace4-49e9-ba67-b1a9ec34a420 admin admin] PATH_INFO: `/v3/users/b74f6c9c99824e22a599d5cbe5eef46e` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:12.573023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636a835-ace4-49e9-ba67-b1a9ec34a420 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b74f6c9c99824e22a599d5cbe5eef46e)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:12.575771 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636a835-ace4-49e9-ba67-b1a9ec34a420 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:12.751443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7636a835-ace4-49e9-ba67-b1a9ec34a420 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b74f6c9c99824e22a599d5cbe5eef46e'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:12.769235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7636a835-ace4-49e9-ba67-b1a9ec34a420 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b74f6c9c99824e22a599d5cbe5eef46e'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:12.836126 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7cb66-b079-4449-a1ea-13235515c7d2 service swift] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:12.837217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7636a835-ace4-49e9-ba67-b1a9ec34a420 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b74f6c9c99824e22a599d5cbe5eef46e'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:12.837979 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7636a835-ace4-49e9-ba67-b1a9ec34a420 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b74f6c9c99824e22a599d5cbe5eef46e'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:12.838135 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-89c7cb66-b079-4449-a1ea-13235515c7d2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:12.838878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7636a835-ace4-49e9-ba67-b1a9ec34a420 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b74f6c9c99824e22a599d5cbe5eef46e'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:12.845501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:12.860194 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7cb66-b079-4449-a1ea-13235515c7d2 service swift] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:12.874897 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156dfb8-deb5-4247-aaaf-b3d4843ca8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:12.875037 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1144/2311] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:48:12 2022] DELETE /identity/v3/users/b74f6c9c99824e22a599d5cbe5eef46e => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:12.875135 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156dfb8-deb5-4247-aaaf-b3d4843ca8cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:12.875135 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156dfb8-deb5-4247-aaaf-b3d4843ca8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:12.875135 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156dfb8-deb5-4247-aaaf-b3d4843ca8cd admin admin] PATH_INFO: `/v3/users/b2b4a938daa0464a9be11627edf0e6d9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:12.875135 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156dfb8-deb5-4247-aaaf-b3d4843ca8cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2b4a938daa0464a9be11627edf0e6d9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:12.875135 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156dfb8-deb5-4247-aaaf-b3d4843ca8cd admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:12.875674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e17736-1fb4-4d1d-b18a-3e16ded555cc None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:12.876097 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e17736-1fb4-4d1d-b18a-3e16ded555cc None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:12.876377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e17736-1fb4-4d1d-b18a-3e16ded555cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:12.908509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-38e17736-1fb4-4d1d-b18a-3e16ded555cc None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:12.910017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-38e17736-1fb4-4d1d-b18a-3e16ded555cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:12.999407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1145/2312] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:48:13.011824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e17736-1fb4-4d1d-b18a-3e16ded555cc None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:13.013859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-38e17736-1fb4-4d1d-b18a-3e16ded555cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:13.077240 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e156dfb8-deb5-4247-aaaf-b3d4843ca8cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2b4a938daa0464a9be11627edf0e6d9'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:13.094275 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:13.099929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e156dfb8-deb5-4247-aaaf-b3d4843ca8cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2b4a938daa0464a9be11627edf0e6d9'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:13.100924 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e156dfb8-deb5-4247-aaaf-b3d4843ca8cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2b4a938daa0464a9be11627edf0e6d9'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:13.101593 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e156dfb8-deb5-4247-aaaf-b3d4843ca8cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2b4a938daa0464a9be11627edf0e6d9'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:13.120019 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e156dfb8-deb5-4247-aaaf-b3d4843ca8cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2b4a938daa0464a9be11627edf0e6d9'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:13.149744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e17736-1fb4-4d1d-b18a-3e16ded555cc None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:13.162559 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbb7ec-2079-4f0d-891c-61133f972c95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:13.163336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbb7ec-2079-4f0d-891c-61133f972c95 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:13.163620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbb7ec-2079-4f0d-891c-61133f972c95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:13.163890 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbb7ec-2079-4f0d-891c-61133f972c95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:13.166628 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-6cbbb7ec-2079-4f0d-891c-61133f972c95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:13.181691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbb7ec-2079-4f0d-891c-61133f972c95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:13.183745 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbb7ec-2079-4f0d-891c-61133f972c95 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:13.196902 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1146/2313] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:48:13.197664 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbb7ec-2079-4f0d-891c-61133f972c95 service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:13.200981 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-6cbbb7ec-2079-4f0d-891c-61133f972c95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:13.214625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbb7ec-2079-4f0d-891c-61133f972c95 service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:13.224603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1168/2314] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:48:12 2022] DELETE /identity/v3/users/b2b4a938daa0464a9be11627edf0e6d9 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:13.226116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346b6a3-e9a4-4c04-9489-38e00191504c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:13.226361 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37981173-621f-4d98-a4c3-d352073fe883 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:13.226697 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37981173-621f-4d98-a4c3-d352073fe883 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:13.227008 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37981173-621f-4d98-a4c3-d352073fe883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:13.228088 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346b6a3-e9a4-4c04-9489-38e00191504c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:13.229643 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346b6a3-e9a4-4c04-9489-38e00191504c admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:13.232069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346b6a3-e9a4-4c04-9489-38e00191504c admin admin] PATH_INFO: `/v3/users/3e47682645964d1797d13ddb62003bac` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:13.234967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346b6a3-e9a4-4c04-9489-38e00191504c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e47682645964d1797d13ddb62003bac)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:13.240319 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346b6a3-e9a4-4c04-9489-38e00191504c admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:13.251860 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-37981173-621f-4d98-a4c3-d352073fe883 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:13.252513 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-37981173-621f-4d98-a4c3-d352073fe883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:13.352839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1169/2315] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:48:13.360089 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37981173-621f-4d98-a4c3-d352073fe883 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:13.361656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-37981173-621f-4d98-a4c3-d352073fe883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:13.448410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37981173-621f-4d98-a4c3-d352073fe883 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:13.475296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-e346b6a3-e9a4-4c04-9489-38e00191504c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e47682645964d1797d13ddb62003bac'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:13.482495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a05ada-4c4a-469e-928d-d09c672ac5cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:13.483205 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a05ada-4c4a-469e-928d-d09c672ac5cb service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:13.483494 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a05ada-4c4a-469e-928d-d09c672ac5cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:13.488881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a05ada-4c4a-469e-928d-d09c672ac5cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:13.509696 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a05ada-4c4a-469e-928d-d09c672ac5cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:13.512608 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a05ada-4c4a-469e-928d-d09c672ac5cb service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:13.513158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-e346b6a3-e9a4-4c04-9489-38e00191504c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e47682645964d1797d13ddb62003bac'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:13.531350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1170/2316] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:48:13.608431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-e346b6a3-e9a4-4c04-9489-38e00191504c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e47682645964d1797d13ddb62003bac'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:13.609905 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-e346b6a3-e9a4-4c04-9489-38e00191504c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e47682645964d1797d13ddb62003bac'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:13.610583 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-e346b6a3-e9a4-4c04-9489-38e00191504c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e47682645964d1797d13ddb62003bac'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:13.633022 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1147/2317] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:48:13 2022] DELETE /identity/v3/users/3e47682645964d1797d13ddb62003bac => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:13.638329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd882c17-cef8-41fb-8531-c0795dcf9492 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:13.638624 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd882c17-cef8-41fb-8531-c0795dcf9492 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:13.638908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd882c17-cef8-41fb-8531-c0795dcf9492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:13.664746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-cd882c17-cef8-41fb-8531-c0795dcf9492 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:13.664746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-cd882c17-cef8-41fb-8531-c0795dcf9492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:13.747162 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1171/2318] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:13.758958 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346b6a3-e9a4-4c04-9489-38e00191504c admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:13.760803 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e346b6a3-e9a4-4c04-9489-38e00191504c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:13.768130 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd882c17-cef8-41fb-8531-c0795dcf9492 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:13.769772 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-cd882c17-cef8-41fb-8531-c0795dcf9492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:13.803183 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346b6a3-e9a4-4c04-9489-38e00191504c admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:13.828754 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01300378-07cc-4bb4-b34b-e59118ed6c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:13.842352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01300378-07cc-4bb4-b34b-e59118ed6c07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:13.845425 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01300378-07cc-4bb4-b34b-e59118ed6c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:13.846614 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01300378-07cc-4bb4-b34b-e59118ed6c07 admin admin] PATH_INFO: `/v3/projects/ebac7a1ed527462fbb0ab91d84bf5771` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:13.848425 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01300378-07cc-4bb4-b34b-e59118ed6c07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebac7a1ed527462fbb0ab91d84bf5771)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:13.852507 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01300378-07cc-4bb4-b34b-e59118ed6c07 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:13.856421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:13.868480 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:13.939518 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd882c17-cef8-41fb-8531-c0795dcf9492 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:13.947578 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab51e36-ea05-44dc-b932-110706d4f5c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:13.949259 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab51e36-ea05-44dc-b932-110706d4f5c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:13.949734 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab51e36-ea05-44dc-b932-110706d4f5c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:13.950099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab51e36-ea05-44dc-b932-110706d4f5c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:13.962267 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab51e36-ea05-44dc-b932-110706d4f5c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:13.963831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab51e36-ea05-44dc-b932-110706d4f5c6 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:13.974739 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1172/2319] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:48:14.037805 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-01300378-07cc-4bb4-b34b-e59118ed6c07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebac7a1ed527462fbb0ab91d84bf5771', 'request_id': 'req-01300378-07cc-4bb4-b34b-e59118ed6c07'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:14.044961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab51e36-ea05-44dc-b932-110706d4f5c6 service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:14.049814 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-1ab51e36-ea05-44dc-b932-110706d4f5c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:14.062600 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab51e36-ea05-44dc-b932-110706d4f5c6 service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:14.184205 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213df56-e2ab-4f51-a41c-b898d7a1f5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:14.196635 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213df56-e2ab-4f51-a41c-b898d7a1f5fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:14.196635 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213df56-e2ab-4f51-a41c-b898d7a1f5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:14.196635 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213df56-e2ab-4f51-a41c-b898d7a1f5fe admin admin] PATH_INFO: `/v3/projects/81bfe73155c0422fb29425bc4b3ae4ff` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:14.196635 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213df56-e2ab-4f51-a41c-b898d7a1f5fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=81bfe73155c0422fb29425bc4b3ae4ff)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:14.196635 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213df56-e2ab-4f51-a41c-b898d7a1f5fe admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:14.273451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-01300378-07cc-4bb4-b34b-e59118ed6c07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebac7a1ed527462fbb0ab91d84bf5771', 'request_id': 'req-01300378-07cc-4bb4-b34b-e59118ed6c07'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:14.282006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-01300378-07cc-4bb4-b34b-e59118ed6c07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebac7a1ed527462fbb0ab91d84bf5771', 'request_id': 'req-01300378-07cc-4bb4-b34b-e59118ed6c07'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:14.312680 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-01300378-07cc-4bb4-b34b-e59118ed6c07 admin admin] The token cache is being invalidate because project ebac7a1ed527462fbb0ab91d84bf5771 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:48:14.312680 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-01300378-07cc-4bb4-b34b-e59118ed6c07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:14.312680 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-01300378-07cc-4bb4-b34b-e59118ed6c07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:14.312680 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1148/2320] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:48:13 2022] DELETE /identity/v3/projects/ebac7a1ed527462fbb0ab91d84bf5771 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:14.317091 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01300378-07cc-4bb4-b34b-e59118ed6c07 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:14.321224 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-01300378-07cc-4bb4-b34b-e59118ed6c07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:14.340768 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01300378-07cc-4bb4-b34b-e59118ed6c07 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:14.346209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442fc2d-e6a8-4043-8bd2-fec76a43b324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:14.347214 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442fc2d-e6a8-4043-8bd2-fec76a43b324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:14.347507 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442fc2d-e6a8-4043-8bd2-fec76a43b324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:14.347791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442fc2d-e6a8-4043-8bd2-fec76a43b324 admin admin] PATH_INFO: `/v3/users/c6b4ce1be2ed45438fce9d7ed798d64d` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:14.349702 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f442fc2d-e6a8-4043-8bd2-fec76a43b324 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6b4ce1be2ed45438fce9d7ed798d64d)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:14.351695 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f442fc2d-e6a8-4043-8bd2-fec76a43b324 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:14.780250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c213df56-e2ab-4f51-a41c-b898d7a1f5fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81bfe73155c0422fb29425bc4b3ae4ff', 'request_id': 'req-c213df56-e2ab-4f51-a41c-b898d7a1f5fe'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:14.788680 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c213df56-e2ab-4f51-a41c-b898d7a1f5fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81bfe73155c0422fb29425bc4b3ae4ff', 'request_id': 'req-c213df56-e2ab-4f51-a41c-b898d7a1f5fe'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:14.792822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c213df56-e2ab-4f51-a41c-b898d7a1f5fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81bfe73155c0422fb29425bc4b3ae4ff', 'request_id': 'req-c213df56-e2ab-4f51-a41c-b898d7a1f5fe'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:14.882632 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f442fc2d-e6a8-4043-8bd2-fec76a43b324 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6b4ce1be2ed45438fce9d7ed798d64d'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:14.902043 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c213df56-e2ab-4f51-a41c-b898d7a1f5fe admin admin] The token cache is being invalidate because project 81bfe73155c0422fb29425bc4b3ae4ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:48:14.908860 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c213df56-e2ab-4f51-a41c-b898d7a1f5fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:14.908860 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c213df56-e2ab-4f51-a41c-b898d7a1f5fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:14.908860 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1173/2321] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:48:14 2022] DELETE /identity/v3/projects/81bfe73155c0422fb29425bc4b3ae4ff => generated 0 bytes in 862 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:48:14.912106 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e626d-243b-4002-ad13-e3998dc3d200 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:14.912514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e626d-243b-4002-ad13-e3998dc3d200 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:14.912895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e626d-243b-4002-ad13-e3998dc3d200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:14.920488 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f442fc2d-e6a8-4043-8bd2-fec76a43b324 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6b4ce1be2ed45438fce9d7ed798d64d'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:14.962626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-d67e626d-243b-4002-ad13-e3998dc3d200 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:14.966203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d67e626d-243b-4002-ad13-e3998dc3d200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:14.985800 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f442fc2d-e6a8-4043-8bd2-fec76a43b324 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6b4ce1be2ed45438fce9d7ed798d64d'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:14.986436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f442fc2d-e6a8-4043-8bd2-fec76a43b324 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6b4ce1be2ed45438fce9d7ed798d64d'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:14.987166 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f442fc2d-e6a8-4043-8bd2-fec76a43b324 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6b4ce1be2ed45438fce9d7ed798d64d'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:15.022300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1149/2322] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:48:14 2022] DELETE /identity/v3/users/c6b4ce1be2ed45438fce9d7ed798d64d => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:15.032707 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442fc2d-e6a8-4043-8bd2-fec76a43b324 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:15.034851 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-f442fc2d-e6a8-4043-8bd2-fec76a43b324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:15.041281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:15.069141 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1174/2323] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:48:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:48:15.075226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d734d14-0b54-4240-879a-f5da34178236 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:15.075226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d734d14-0b54-4240-879a-f5da34178236 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:15.075226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d734d14-0b54-4240-879a-f5da34178236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:15.095815 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-0d734d14-0b54-4240-879a-f5da34178236 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:15.097164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-0d734d14-0b54-4240-879a-f5da34178236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:15.131401 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442fc2d-e6a8-4043-8bd2-fec76a43b324 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:15.139108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd4a4a-0fc4-400d-a212-87739e611ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:15.140682 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd4a4a-0fc4-400d-a212-87739e611ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:15.141352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd4a4a-0fc4-400d-a212-87739e611ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:15.141835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd4a4a-0fc4-400d-a212-87739e611ce1 admin admin] PATH_INFO: `/v3/projects/7b88b05dded7480d8f825e3c320fbc8d` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:15.143210 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd4a4a-0fc4-400d-a212-87739e611ce1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b88b05dded7480d8f825e3c320fbc8d)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:15.145695 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd4a4a-0fc4-400d-a212-87739e611ce1 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:15.157418 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:15.215951 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1175/2324] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:15.220177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d734d14-0b54-4240-879a-f5da34178236 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:15.230992 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d734d14-0b54-4240-879a-f5da34178236 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:15.236181 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90701ff5-7def-4b5c-862a-0dbc53cfa590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:15.237022 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90701ff5-7def-4b5c-862a-0dbc53cfa590 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:15.237382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90701ff5-7def-4b5c-862a-0dbc53cfa590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:15.237738 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90701ff5-7def-4b5c-862a-0dbc53cfa590 admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:15.238591 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90701ff5-7def-4b5c-862a-0dbc53cfa590 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:48:15.238989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90701ff5-7def-4b5c-862a-0dbc53cfa590 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:15.242075 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90701ff5-7def-4b5c-862a-0dbc53cfa590 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:15.259997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1176/2325] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:48:15.264295 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90701ff5-7def-4b5c-862a-0dbc53cfa590 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:15.281407 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90701ff5-7def-4b5c-862a-0dbc53cfa590 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:15.305099 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-92cd4a4a-0fc4-400d-a212-87739e611ce1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b88b05dded7480d8f825e3c320fbc8d', 'request_id': 'req-92cd4a4a-0fc4-400d-a212-87739e611ce1'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:15.382434 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-92cd4a4a-0fc4-400d-a212-87739e611ce1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b88b05dded7480d8f825e3c320fbc8d', 'request_id': 'req-92cd4a4a-0fc4-400d-a212-87739e611ce1'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:15.401524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff354f1b-7222-4bf4-8852-b2d2c128c896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:15.402621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff354f1b-7222-4bf4-8852-b2d2c128c896 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:15.403086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff354f1b-7222-4bf4-8852-b2d2c128c896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:15.403378 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff354f1b-7222-4bf4-8852-b2d2c128c896 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:15.404209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff354f1b-7222-4bf4-8852-b2d2c128c896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:15.405460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-92cd4a4a-0fc4-400d-a212-87739e611ce1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b88b05dded7480d8f825e3c320fbc8d', 'request_id': 'req-92cd4a4a-0fc4-400d-a212-87739e611ce1'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:15.406910 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff354f1b-7222-4bf4-8852-b2d2c128c896 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:15.437541 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-92cd4a4a-0fc4-400d-a212-87739e611ce1 admin admin] The token cache is being invalidate because project 7b88b05dded7480d8f825e3c320fbc8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:48:15.438020 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-92cd4a4a-0fc4-400d-a212-87739e611ce1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:15.441563 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-92cd4a4a-0fc4-400d-a212-87739e611ce1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:15.450968 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1150/2326] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:48:15 2022] DELETE /identity/v3/projects/7b88b05dded7480d8f825e3c320fbc8d => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:15.452883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752036d-2e2f-43dc-99d1-ce81782ce209 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:15.453439 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752036d-2e2f-43dc-99d1-ce81782ce209 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:15.453842 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752036d-2e2f-43dc-99d1-ce81782ce209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:15.480218 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-0752036d-2e2f-43dc-99d1-ce81782ce209 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:15.481624 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-0752036d-2e2f-43dc-99d1-ce81782ce209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:15.490732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1151/2327] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:15.500144 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cb2c3-2fc2-4817-bd86-4c2c528682d9 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:15.500412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cb2c3-2fc2-4817-bd86-4c2c528682d9 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:15.500842 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cb2c3-2fc2-4817-bd86-4c2c528682d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:15.506431 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1177/2328] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:48:15 2022] POST /identity/v3/projects => generated 397 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:48:15.507624 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff354f1b-7222-4bf4-8852-b2d2c128c896 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:15.510509 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ff354f1b-7222-4bf4-8852-b2d2c128c896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:15.538452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-792cb2c3-2fc2-4817-bd86-4c2c528682d9 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:15.542646 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-792cb2c3-2fc2-4817-bd86-4c2c528682d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:15.645633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1152/2329] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:48:15.648569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792cb2c3-2fc2-4817-bd86-4c2c528682d9 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:15.669194 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff354f1b-7222-4bf4-8852-b2d2c128c896 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:15.672730 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792cb2c3-2fc2-4817-bd86-4c2c528682d9 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:15.682565 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5eb6a9-5569-4de1-8d44-0e9d4b0f1084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:15.683470 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5eb6a9-5569-4de1-8d44-0e9d4b0f1084 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:15.683760 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5eb6a9-5569-4de1-8d44-0e9d4b0f1084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:15.684123 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5eb6a9-5569-4de1-8d44-0e9d4b0f1084 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:15.685075 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5eb6a9-5569-4de1-8d44-0e9d4b0f1084 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:15.686260 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384666a-e8bf-4b02-94ad-ea996c07c03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:15.687727 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384666a-e8bf-4b02-94ad-ea996c07c03c admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:15.687727 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384666a-e8bf-4b02-94ad-ea996c07c03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:15.687925 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5eb6a9-5569-4de1-8d44-0e9d4b0f1084 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:15.688022 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384666a-e8bf-4b02-94ad-ea996c07c03c admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:15.688858 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384666a-e8bf-4b02-94ad-ea996c07c03c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:48:15.689278 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384666a-e8bf-4b02-94ad-ea996c07c03c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:15.690996 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384666a-e8bf-4b02-94ad-ea996c07c03c admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:15.709799 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1153/2330] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:48:15.711220 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384666a-e8bf-4b02-94ad-ea996c07c03c admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:15.724781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384666a-e8bf-4b02-94ad-ea996c07c03c admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:15.729843 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c739fd5-e959-43ba-b4a7-258f4770353a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:15.730538 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c739fd5-e959-43ba-b4a7-258f4770353a admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:15.730826 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c739fd5-e959-43ba-b4a7-258f4770353a admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:15.731159 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c739fd5-e959-43ba-b4a7-258f4770353a admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:15.731858 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c739fd5-e959-43ba-b4a7-258f4770353a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:15.733907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c739fd5-e959-43ba-b4a7-258f4770353a admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:15.770462 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:15.779629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1178/2331] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:48:15 2022] POST /identity/v3/users => generated 299 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:48:15.779629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5eb6a9-5569-4de1-8d44-0e9d4b0f1084 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:15.784787 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3b5eb6a9-5569-4de1-8d44-0e9d4b0f1084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:15.800940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5eb6a9-5569-4de1-8d44-0e9d4b0f1084 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:15.818712 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda216af-a6a6-45b4-8e6f-f87761427f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:15.819812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda216af-a6a6-45b4-8e6f-f87761427f83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:15.820286 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda216af-a6a6-45b4-8e6f-f87761427f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:15.820772 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda216af-a6a6-45b4-8e6f-f87761427f83 admin admin] PATH_INFO: `/v3/projects/1c3c2031ab43423bb8467b4005ba401c` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:15.822173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda216af-a6a6-45b4-8e6f-f87761427f83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c3c2031ab43423bb8467b4005ba401c)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:15.824102 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda216af-a6a6-45b4-8e6f-f87761427f83 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:15.846917 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1154/2332] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:48:15 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:48:15.848404 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c739fd5-e959-43ba-b4a7-258f4770353a admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:15.911062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:15.924881 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c739fd5-e959-43ba-b4a7-258f4770353a admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:15.933524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13103a9d-99e2-4725-adbe-2c0498b7bcad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:15.935435 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13103a9d-99e2-4725-adbe-2c0498b7bcad admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:15.935951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13103a9d-99e2-4725-adbe-2c0498b7bcad admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:15.936526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13103a9d-99e2-4725-adbe-2c0498b7bcad admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:15.937340 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13103a9d-99e2-4725-adbe-2c0498b7bcad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:15.939318 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13103a9d-99e2-4725-adbe-2c0498b7bcad admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:15.949556 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-eda216af-a6a6-45b4-8e6f-f87761427f83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c3c2031ab43423bb8467b4005ba401c', 'request_id': 'req-eda216af-a6a6-45b4-8e6f-f87761427f83'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:15.950842 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1155/2333] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:48:15.952176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13103a9d-99e2-4725-adbe-2c0498b7bcad admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:15.958077 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-eda216af-a6a6-45b4-8e6f-f87761427f83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c3c2031ab43423bb8467b4005ba401c', 'request_id': 'req-eda216af-a6a6-45b4-8e6f-f87761427f83'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:15.965933 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-eda216af-a6a6-45b4-8e6f-f87761427f83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c3c2031ab43423bb8467b4005ba401c', 'request_id': 'req-eda216af-a6a6-45b4-8e6f-f87761427f83'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:15.974557 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13103a9d-99e2-4725-adbe-2c0498b7bcad admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:16.037005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-eda216af-a6a6-45b4-8e6f-f87761427f83 admin admin] The token cache is being invalidate because project 1c3c2031ab43423bb8467b4005ba401c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:48:16.037296 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-eda216af-a6a6-45b4-8e6f-f87761427f83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:16.037963 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-eda216af-a6a6-45b4-8e6f-f87761427f83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:16.039944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1179/2334] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:48:15 2022] DELETE /identity/v3/projects/1c3c2031ab43423bb8467b4005ba401c => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:16.040849 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda216af-a6a6-45b4-8e6f-f87761427f83 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:16.043220 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-eda216af-a6a6-45b4-8e6f-f87761427f83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:16.087879 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7a223-4a81-4d08-a04c-301c6717cb09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:16.088616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7a223-4a81-4d08-a04c-301c6717cb09 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:16.089058 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7a223-4a81-4d08-a04c-301c6717cb09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:16.089351 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7a223-4a81-4d08-a04c-301c6717cb09 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:16.090089 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7a223-4a81-4d08-a04c-301c6717cb09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:16.100428 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7a223-4a81-4d08-a04c-301c6717cb09 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:16.140330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda216af-a6a6-45b4-8e6f-f87761427f83 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:16.153835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc5795-6399-47b4-afea-937ddf553a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:16.162317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc5795-6399-47b4-afea-937ddf553a88 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:16.162317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc5795-6399-47b4-afea-937ddf553a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:16.162317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc5795-6399-47b4-afea-937ddf553a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:16.162317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc5795-6399-47b4-afea-937ddf553a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:16.162317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc5795-6399-47b4-afea-937ddf553a88 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:16.191184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1156/2335] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:48:15 2022] POST /identity/v3/users => generated 293 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:48:16.192569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7a223-4a81-4d08-a04c-301c6717cb09 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:16.195947 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-18c7a223-4a81-4d08-a04c-301c6717cb09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:16.203121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1180/2336] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:48:16.212345 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc5795-6399-47b4-afea-937ddf553a88 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:16.225972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7a223-4a81-4d08-a04c-301c6717cb09 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:16.231619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7a7d9-6d2b-4fc2-9e6e-9275ba8f80c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:16.232318 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7a7d9-6d2b-4fc2-9e6e-9275ba8f80c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:16.232632 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7a7d9-6d2b-4fc2-9e6e-9275ba8f80c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:16.233095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7a7d9-6d2b-4fc2-9e6e-9275ba8f80c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:16.233851 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7a7d9-6d2b-4fc2-9e6e-9275ba8f80c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:16.235620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7a7d9-6d2b-4fc2-9e6e-9275ba8f80c5 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:16.236967 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc5795-6399-47b4-afea-937ddf553a88 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:16.248230 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f405a8-d6e0-4fc4-a2bd-779f3663dddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:16.252140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1157/2337] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:48:16.253611 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7a7d9-6d2b-4fc2-9e6e-9275ba8f80c5 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:16.256003 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f405a8-d6e0-4fc4-a2bd-779f3663dddf admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:16.256953 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f405a8-d6e0-4fc4-a2bd-779f3663dddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:16.264350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f405a8-d6e0-4fc4-a2bd-779f3663dddf admin admin] PATH_INFO: `/v3/projects/c30b5f16088e42a48d5183baa55902bf/users/11752a1c73ef4114ab37bef7bab4662f/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:16.272301 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7a7d9-6d2b-4fc2-9e6e-9275ba8f80c5 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:16.277965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657ddab-8e9e-4fea-819c-4d05122f207f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:16.278769 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657ddab-8e9e-4fea-819c-4d05122f207f admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:16.279159 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657ddab-8e9e-4fea-819c-4d05122f207f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:16.279539 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657ddab-8e9e-4fea-819c-4d05122f207f admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:16.280350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657ddab-8e9e-4fea-819c-4d05122f207f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:16.288274 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657ddab-8e9e-4fea-819c-4d05122f207f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:16.303923 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1158/2338] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:48:16.305602 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657ddab-8e9e-4fea-819c-4d05122f207f admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:16.316915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f405a8-d6e0-4fc4-a2bd-779f3663dddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c30b5f16088e42a48d5183baa55902bf, user_id=11752a1c73ef4114ab37bef7bab4662f, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:16.320473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f405a8-d6e0-4fc4-a2bd-779f3663dddf admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:16.326860 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657ddab-8e9e-4fea-819c-4d05122f207f admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:16.335076 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b34d1b-20c0-4f06-a87e-a53155792f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:16.335984 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b34d1b-20c0-4f06-a87e-a53155792f0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:16.336294 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b34d1b-20c0-4f06-a87e-a53155792f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:16.336571 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b34d1b-20c0-4f06-a87e-a53155792f0b admin admin] PATH_INFO: `/v3/projects/f58b51c969054b48ba49d533b4e93cb5/users/4905be024e374597a13cdc4ef214e0d2/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:16.357611 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b34d1b-20c0-4f06-a87e-a53155792f0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f58b51c969054b48ba49d533b4e93cb5, user_id=4905be024e374597a13cdc4ef214e0d2, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:16.360368 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b34d1b-20c0-4f06-a87e-a53155792f0b admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:16.415998 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1181/2339] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:48:16 2022] PUT /identity/v3/projects/c30b5f16088e42a48d5183baa55902bf/users/11752a1c73ef4114ab37bef7bab4662f/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:16.419781 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c947a-15f0-4257-8883-6949b4439011 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:16.421858 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c947a-15f0-4257-8883-6949b4439011 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:16.422196 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c947a-15f0-4257-8883-6949b4439011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:16.434247 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1159/2340] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:48:16 2022] PUT /identity/v3/projects/f58b51c969054b48ba49d533b4e93cb5/users/4905be024e374597a13cdc4ef214e0d2/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:16.452749 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e736c2-5eea-442e-9671-e4a84ebae5c6 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:16.452749 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e736c2-5eea-442e-9671-e4a84ebae5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:16.452749 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e736c2-5eea-442e-9671-e4a84ebae5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:16.496962 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-f18c947a-15f0-4257-8883-6949b4439011 None None] MFA Rules not processed for user `11752a1c73ef4114ab37bef7bab4662f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:16.498913 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-f18c947a-15f0-4257-8883-6949b4439011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:16.508179 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-39e736c2-5eea-442e-9671-e4a84ebae5c6 None None] MFA Rules not processed for user `4905be024e374597a13cdc4ef214e0d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:16.510342 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-39e736c2-5eea-442e-9671-e4a84ebae5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:16.576774 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:16.613193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1182/2341] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:48:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:16.626085 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c947a-15f0-4257-8883-6949b4439011 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:16.627856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-f18c947a-15f0-4257-8883-6949b4439011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:16.636410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1160/2342] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:48:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:16.644749 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e736c2-5eea-442e-9671-e4a84ebae5c6 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:16.652102 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-39e736c2-5eea-442e-9671-e4a84ebae5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:16.690397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:16.727331 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c947a-15f0-4257-8883-6949b4439011 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:16.734413 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e736c2-5eea-442e-9671-e4a84ebae5c6 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:16.736315 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10bb80-8211-4894-aaab-f1f1e4c03211 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:16.737129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10bb80-8211-4894-aaab-f1f1e4c03211 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:16.737509 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10bb80-8211-4894-aaab-f1f1e4c03211 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:16.737789 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10bb80-8211-4894-aaab-f1f1e4c03211 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:16.738719 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cb6c9-50b0-4309-a525-4a93a392b6c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:16.739974 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cb6c9-50b0-4309-a525-4a93a392b6c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:16.740600 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cb6c9-50b0-4309-a525-4a93a392b6c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:16.741032 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cb6c9-50b0-4309-a525-4a93a392b6c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:16.749296 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10bb80-8211-4894-aaab-f1f1e4c03211 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:16.758457 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cb6c9-50b0-4309-a525-4a93a392b6c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:16.760162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cb6c9-50b0-4309-a525-4a93a392b6c7 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:16.763490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10bb80-8211-4894-aaab-f1f1e4c03211 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:16.773961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1183/2343] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:48:16.787036 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1161/2344] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:48:17.852047 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10bb80-8211-4894-aaab-f1f1e4c03211 service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:17.853629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-1d10bb80-8211-4894-aaab-f1f1e4c03211 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:18.041164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10bb80-8211-4894-aaab-f1f1e4c03211 service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:18.052652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944a5ee-0965-49db-83b4-a1719f1a19a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ac44b3f0224e56a15993a2f29c1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:18.053560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944a5ee-0965-49db-83b4-a1719f1a19a3 service placement] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:18.053851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944a5ee-0965-49db-83b4-a1719f1a19a3 service placement] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:18.054322 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944a5ee-0965-49db-83b4-a1719f1a19a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:18.056165 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-9944a5ee-0965-49db-83b4-a1719f1a19a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:18.074684 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944a5ee-0965-49db-83b4-a1719f1a19a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:18.076373 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944a5ee-0965-49db-83b4-a1719f1a19a3 service placement] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:18.092476 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1184/2345] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:48:21.243523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cb6c9-50b0-4309-a525-4a93a392b6c7 service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:21.314229 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cb6c9-50b0-4309-a525-4a93a392b6c7 service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:21.319455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5657c-02d6-4da4-8229-1fed3e59476d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:21.320367 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5657c-02d6-4da4-8229-1fed3e59476d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:21.320684 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5657c-02d6-4da4-8229-1fed3e59476d admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:21.320965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5657c-02d6-4da4-8229-1fed3e59476d admin admin] PATH_INFO: `/v3/users/4905be024e374597a13cdc4ef214e0d2` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:21.322143 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5657c-02d6-4da4-8229-1fed3e59476d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4905be024e374597a13cdc4ef214e0d2)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:21.324067 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5657c-02d6-4da4-8229-1fed3e59476d admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:21.518582 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-8ca5657c-02d6-4da4-8229-1fed3e59476d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4905be024e374597a13cdc4ef214e0d2'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:21.535046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-8ca5657c-02d6-4da4-8229-1fed3e59476d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4905be024e374597a13cdc4ef214e0d2'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:21.625117 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-8ca5657c-02d6-4da4-8229-1fed3e59476d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4905be024e374597a13cdc4ef214e0d2'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:21.625813 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-8ca5657c-02d6-4da4-8229-1fed3e59476d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4905be024e374597a13cdc4ef214e0d2'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:21.626426 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-8ca5657c-02d6-4da4-8229-1fed3e59476d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4905be024e374597a13cdc4ef214e0d2'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:21.640238 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1162/2346] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:48:21 2022] DELETE /identity/v3/users/4905be024e374597a13cdc4ef214e0d2 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:21.644857 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27df03-d6fc-43fa-8b78-4fe12207823d None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:21.646090 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27df03-d6fc-43fa-8b78-4fe12207823d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:21.646090 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27df03-d6fc-43fa-8b78-4fe12207823d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:21.664841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-0b27df03-d6fc-43fa-8b78-4fe12207823d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:21.666713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-0b27df03-d6fc-43fa-8b78-4fe12207823d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:21.727159 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:21.744711 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1185/2347] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:21.752115 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5657c-02d6-4da4-8229-1fed3e59476d admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:21.753577 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-8ca5657c-02d6-4da4-8229-1fed3e59476d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:21.802845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:21.813440 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5657c-02d6-4da4-8229-1fed3e59476d admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:21.817145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7df069-1dc9-4d84-bfc0-7948cc61736b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:21.817878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7df069-1dc9-4d84-bfc0-7948cc61736b service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:21.818166 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7df069-1dc9-4d84-bfc0-7948cc61736b service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:21.818563 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7df069-1dc9-4d84-bfc0-7948cc61736b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:21.828055 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7df069-1dc9-4d84-bfc0-7948cc61736b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:21.829647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7df069-1dc9-4d84-bfc0-7948cc61736b service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:21.839919 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1163/2348] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:48:22.261240 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27df03-d6fc-43fa-8b78-4fe12207823d None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:22.262817 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-0b27df03-d6fc-43fa-8b78-4fe12207823d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:22.285354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27df03-d6fc-43fa-8b78-4fe12207823d None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:22.292191 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8f1e5-edf9-4307-b68f-d73a0daebb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:22.293334 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8f1e5-edf9-4307-b68f-d73a0daebb02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:22.294417 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8f1e5-edf9-4307-b68f-d73a0daebb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:22.294417 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8f1e5-edf9-4307-b68f-d73a0daebb02 admin admin] PATH_INFO: `/v3/projects/f58b51c969054b48ba49d533b4e93cb5` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:22.295633 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf8f1e5-edf9-4307-b68f-d73a0daebb02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f58b51c969054b48ba49d533b4e93cb5)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:22.298583 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf8f1e5-edf9-4307-b68f-d73a0daebb02 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:22.434506 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-4bf8f1e5-edf9-4307-b68f-d73a0daebb02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f58b51c969054b48ba49d533b4e93cb5', 'request_id': 'req-4bf8f1e5-edf9-4307-b68f-d73a0daebb02'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:22.441175 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-4bf8f1e5-edf9-4307-b68f-d73a0daebb02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f58b51c969054b48ba49d533b4e93cb5', 'request_id': 'req-4bf8f1e5-edf9-4307-b68f-d73a0daebb02'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:22.447598 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-4bf8f1e5-edf9-4307-b68f-d73a0daebb02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f58b51c969054b48ba49d533b4e93cb5', 'request_id': 'req-4bf8f1e5-edf9-4307-b68f-d73a0daebb02'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:22.529511 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-4bf8f1e5-edf9-4307-b68f-d73a0daebb02 admin admin] The token cache is being invalidate because project f58b51c969054b48ba49d533b4e93cb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:48:22.529815 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-4bf8f1e5-edf9-4307-b68f-d73a0daebb02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:22.530476 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-4bf8f1e5-edf9-4307-b68f-d73a0daebb02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:22.532486 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1186/2349] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:48:22 2022] DELETE /identity/v3/projects/f58b51c969054b48ba49d533b4e93cb5 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:22.538245 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fcd39-af9c-47a5-8f0c-d8be6bfc4e8b None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:22.538691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fcd39-af9c-47a5-8f0c-d8be6bfc4e8b None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:22.538807 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fcd39-af9c-47a5-8f0c-d8be6bfc4e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:22.558622 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-978fcd39-af9c-47a5-8f0c-d8be6bfc4e8b None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:22.559849 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-978fcd39-af9c-47a5-8f0c-d8be6bfc4e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:22.631557 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1164/2350] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:48:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:22.639317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69de7f-fbfb-46cb-a883-ec1f03839460 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:22.639633 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69de7f-fbfb-46cb-a883-ec1f03839460 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:22.639913 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69de7f-fbfb-46cb-a883-ec1f03839460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:22.658205 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-9a69de7f-fbfb-46cb-a883-ec1f03839460 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:22.659410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-9a69de7f-fbfb-46cb-a883-ec1f03839460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:22.667459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1187/2351] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:48:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:22.671897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fcd39-af9c-47a5-8f0c-d8be6bfc4e8b None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:22.682321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fcd39-af9c-47a5-8f0c-d8be6bfc4e8b None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:22.687087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeef3b4-9e56-4839-83a5-38086eb319da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:22.687749 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeef3b4-9e56-4839-83a5-38086eb319da admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:22.688036 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeef3b4-9e56-4839-83a5-38086eb319da admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:22.688316 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeef3b4-9e56-4839-83a5-38086eb319da admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:22.689117 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeef3b4-9e56-4839-83a5-38086eb319da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:48:22.689431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeef3b4-9e56-4839-83a5-38086eb319da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:22.691047 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeef3b4-9e56-4839-83a5-38086eb319da admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:22.703543 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1165/2352] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:48:22.711649 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69de7f-fbfb-46cb-a883-ec1f03839460 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:22.721102 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69de7f-fbfb-46cb-a883-ec1f03839460 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:22.726065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84366935-0c7b-4dfe-937d-00c3a54514d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:22.726803 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84366935-0c7b-4dfe-937d-00c3a54514d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:22.727162 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84366935-0c7b-4dfe-937d-00c3a54514d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:22.727513 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84366935-0c7b-4dfe-937d-00c3a54514d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:22.728286 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84366935-0c7b-4dfe-937d-00c3a54514d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:22.730321 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84366935-0c7b-4dfe-937d-00c3a54514d7 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:22.861483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1188/2353] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:48:22 2022] POST /identity/v3/projects => generated 389 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:48:22.866679 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeef3b4-9e56-4839-83a5-38086eb319da admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:22.876935 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeef3b4-9e56-4839-83a5-38086eb319da admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:22.933712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdead33-b2cd-4b38-bbca-40ba0c4803d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:22.934359 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdead33-b2cd-4b38-bbca-40ba0c4803d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:22.934648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdead33-b2cd-4b38-bbca-40ba0c4803d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:22.934926 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdead33-b2cd-4b38-bbca-40ba0c4803d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:22.935639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdead33-b2cd-4b38-bbca-40ba0c4803d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:22.937627 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdead33-b2cd-4b38-bbca-40ba0c4803d6 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:23.012666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1166/2354] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:48:22 2022] POST /identity/v3/users => generated 295 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:48:23.016755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84366935-0c7b-4dfe-937d-00c3a54514d7 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:23.027520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84366935-0c7b-4dfe-937d-00c3a54514d7 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:23.032324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0e5eb-077a-482f-8438-08320ddef6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:23.033072 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0e5eb-077a-482f-8438-08320ddef6a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:23.039424 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0e5eb-077a-482f-8438-08320ddef6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:23.039424 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0e5eb-077a-482f-8438-08320ddef6a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:23.039424 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0e5eb-077a-482f-8438-08320ddef6a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:23.039424 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0e5eb-077a-482f-8438-08320ddef6a5 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:23.047075 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1189/2355] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:48:23.051060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdead33-b2cd-4b38-bbca-40ba0c4803d6 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:23.061990 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdead33-b2cd-4b38-bbca-40ba0c4803d6 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:23.066627 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb6c66-a2a9-41a6-90fe-606902370ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:23.070779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb6c66-a2a9-41a6-90fe-606902370ccd admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:23.070779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb6c66-a2a9-41a6-90fe-606902370ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:23.070779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb6c66-a2a9-41a6-90fe-606902370ccd admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:23.070779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bb6c66-a2a9-41a6-90fe-606902370ccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:23.070779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bb6c66-a2a9-41a6-90fe-606902370ccd admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:23.080449 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1167/2356] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:48:23.084491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0e5eb-077a-482f-8438-08320ddef6a5 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:23.094549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0e5eb-077a-482f-8438-08320ddef6a5 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:23.101063 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e80429-65ba-4e18-972b-d09c3c0645d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:23.101063 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e80429-65ba-4e18-972b-d09c3c0645d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:23.101063 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e80429-65ba-4e18-972b-d09c3c0645d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:23.101388 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e80429-65ba-4e18-972b-d09c3c0645d4 admin admin] PATH_INFO: `/v3/projects/07cbbb995a5543a1b861033a56afdaa3/users/a71edb6b0adc49358358910b36c5bb13/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:23.116966 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e80429-65ba-4e18-972b-d09c3c0645d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07cbbb995a5543a1b861033a56afdaa3, user_id=a71edb6b0adc49358358910b36c5bb13, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:23.119691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e80429-65ba-4e18-972b-d09c3c0645d4 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:23.173153 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1190/2357] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:48:23 2022] PUT /identity/v3/projects/07cbbb995a5543a1b861033a56afdaa3/users/a71edb6b0adc49358358910b36c5bb13/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:23.184961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50030014-00cd-445c-b170-73fcd4480707 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:23.185256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50030014-00cd-445c-b170-73fcd4480707 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:23.185539 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50030014-00cd-445c-b170-73fcd4480707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:23.211245 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-50030014-00cd-445c-b170-73fcd4480707 None None] MFA Rules not processed for user `a71edb6b0adc49358358910b36c5bb13`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:23.212440 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-50030014-00cd-445c-b170-73fcd4480707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:23.253078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:23.275133 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1168/2358] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:48:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:23.286408 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e80429-65ba-4e18-972b-d09c3c0645d4 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:23.288118 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-90e80429-65ba-4e18-972b-d09c3c0645d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:23.345380 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e80429-65ba-4e18-972b-d09c3c0645d4 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:23.349691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94934057-aaea-48ad-a2c2-23722b62a0d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:23.350338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94934057-aaea-48ad-a2c2-23722b62a0d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:23.350627 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94934057-aaea-48ad-a2c2-23722b62a0d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:23.350906 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94934057-aaea-48ad-a2c2-23722b62a0d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:23.359203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94934057-aaea-48ad-a2c2-23722b62a0d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:23.360661 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94934057-aaea-48ad-a2c2-23722b62a0d0 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:23.370443 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1191/2359] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:48:29.223407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50030014-00cd-445c-b170-73fcd4480707 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:29.225020 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-50030014-00cd-445c-b170-73fcd4480707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:29.310036 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50030014-00cd-445c-b170-73fcd4480707 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:29.315033 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6c0c9-cfd8-41c5-a3d0-ccabe408732b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:29.316097 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6c0c9-cfd8-41c5-a3d0-ccabe408732b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:29.316382 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6c0c9-cfd8-41c5-a3d0-ccabe408732b admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:29.317320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6c0c9-cfd8-41c5-a3d0-ccabe408732b admin admin] PATH_INFO: `/v3/users/11752a1c73ef4114ab37bef7bab4662f` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:29.318563 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6c0c9-cfd8-41c5-a3d0-ccabe408732b admin admin] RBAC: Authorizing `identity:delete_user(user_id=11752a1c73ef4114ab37bef7bab4662f)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:29.320570 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6c0c9-cfd8-41c5-a3d0-ccabe408732b admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:29.520372 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-2df6c0c9-cfd8-41c5-a3d0-ccabe408732b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11752a1c73ef4114ab37bef7bab4662f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:29.532111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-2df6c0c9-cfd8-41c5-a3d0-ccabe408732b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11752a1c73ef4114ab37bef7bab4662f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:29.601243 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-2df6c0c9-cfd8-41c5-a3d0-ccabe408732b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11752a1c73ef4114ab37bef7bab4662f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:29.601875 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-2df6c0c9-cfd8-41c5-a3d0-ccabe408732b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11752a1c73ef4114ab37bef7bab4662f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:29.604691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-2df6c0c9-cfd8-41c5-a3d0-ccabe408732b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11752a1c73ef4114ab37bef7bab4662f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:29.617230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1169/2360] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:48:29 2022] DELETE /identity/v3/users/11752a1c73ef4114ab37bef7bab4662f => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:29.621827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a4983-3f36-4c4a-8eeb-7d904ca4fb3a None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:29.622332 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a4983-3f36-4c4a-8eeb-7d904ca4fb3a None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:29.622753 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a4983-3f36-4c4a-8eeb-7d904ca4fb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:29.640788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-cf2a4983-3f36-4c4a-8eeb-7d904ca4fb3a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:29.642032 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-cf2a4983-3f36-4c4a-8eeb-7d904ca4fb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:29.662877 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:29.718022 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1192/2361] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:29.728245 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6c0c9-cfd8-41c5-a3d0-ccabe408732b admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:29.729779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-2df6c0c9-cfd8-41c5-a3d0-ccabe408732b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:29.795322 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6c0c9-cfd8-41c5-a3d0-ccabe408732b admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:29.799040 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04082966-f51e-4968-a5b5-4ad88dd4b234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:29.799761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04082966-f51e-4968-a5b5-4ad88dd4b234 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:29.800182 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04082966-f51e-4968-a5b5-4ad88dd4b234 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:29.800295 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04082966-f51e-4968-a5b5-4ad88dd4b234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:29.811518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04082966-f51e-4968-a5b5-4ad88dd4b234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:29.812995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04082966-f51e-4968-a5b5-4ad88dd4b234 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:29.823151 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1170/2362] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:48:30.260531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a4983-3f36-4c4a-8eeb-7d904ca4fb3a None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:30.262004 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-cf2a4983-3f36-4c4a-8eeb-7d904ca4fb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:30.274832 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a4983-3f36-4c4a-8eeb-7d904ca4fb3a None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:30.280676 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61edc42-33dc-4005-b16d-51561df46077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:30.284331 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61edc42-33dc-4005-b16d-51561df46077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:30.284331 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61edc42-33dc-4005-b16d-51561df46077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:30.284331 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61edc42-33dc-4005-b16d-51561df46077 admin admin] PATH_INFO: `/v3/projects/c30b5f16088e42a48d5183baa55902bf` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:30.284331 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61edc42-33dc-4005-b16d-51561df46077 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c30b5f16088e42a48d5183baa55902bf)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:30.286111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61edc42-33dc-4005-b16d-51561df46077 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:30.431813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a61edc42-33dc-4005-b16d-51561df46077 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c30b5f16088e42a48d5183baa55902bf', 'request_id': 'req-a61edc42-33dc-4005-b16d-51561df46077'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:30.438030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a61edc42-33dc-4005-b16d-51561df46077 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c30b5f16088e42a48d5183baa55902bf', 'request_id': 'req-a61edc42-33dc-4005-b16d-51561df46077'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:30.444497 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a61edc42-33dc-4005-b16d-51561df46077 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c30b5f16088e42a48d5183baa55902bf', 'request_id': 'req-a61edc42-33dc-4005-b16d-51561df46077'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:30.532008 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a61edc42-33dc-4005-b16d-51561df46077 admin admin] The token cache is being invalidate because project c30b5f16088e42a48d5183baa55902bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:48:30.532236 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a61edc42-33dc-4005-b16d-51561df46077 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:30.532996 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a61edc42-33dc-4005-b16d-51561df46077 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:30.534614 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1193/2363] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:48:30 2022] DELETE /identity/v3/projects/c30b5f16088e42a48d5183baa55902bf => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:32.438757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7eb19-3bb0-462a-9b65-bbaaf20a5a1f None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:32.439227 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7eb19-3bb0-462a-9b65-bbaaf20a5a1f None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:32.439227 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7eb19-3bb0-462a-9b65-bbaaf20a5a1f None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:32.440652 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1171/2364] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 22:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:48:32.446466 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a6df1-539f-420b-864e-2529626a07ac None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:32.446946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a6df1-539f-420b-864e-2529626a07ac None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:32.447055 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a6df1-539f-420b-864e-2529626a07ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:32.495925 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-3f5a6df1-539f-420b-864e-2529626a07ac None None] MFA Rules not processed for user `159cc8befc3a4d57bbd313832084db62`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:32.497193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3f5a6df1-539f-420b-864e-2529626a07ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:32.574045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1194/2365] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 22:48:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:32.585946 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7eb19-3bb0-462a-9b65-bbaaf20a5a1f None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:32.587604 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-00b7eb19-3bb0-462a-9b65-bbaaf20a5a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:32.593751 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:32.669095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7eb19-3bb0-462a-9b65-bbaaf20a5a1f None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:32.672589 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a95db2-d07c-4934-accd-5ae1b29e65c3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '983801bb18924fe4b3150693765eb13b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:32.673241 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a95db2-d07c-4934-accd-5ae1b29e65c3 service glance] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:32.673528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a95db2-d07c-4934-accd-5ae1b29e65c3 service glance] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:32.673805 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a95db2-d07c-4934-accd-5ae1b29e65c3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:32.684043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a95db2-d07c-4934-accd-5ae1b29e65c3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:32.685508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a95db2-d07c-4934-accd-5ae1b29e65c3 service glance] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:32.697641 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1172/2366] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:48:32.884415 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbac1e3-d2be-4abd-99f1-6faccf1327f5 None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:32.884744 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbac1e3-d2be-4abd-99f1-6faccf1327f5 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:32.885023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbac1e3-d2be-4abd-99f1-6faccf1327f5 None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:32.886387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1195/2367] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 22:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:48:32.891046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c32a0-73f6-4eb9-b92c-3eec61812574 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:32.891536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c32a0-73f6-4eb9-b92c-3eec61812574 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:32.891682 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c32a0-73f6-4eb9-b92c-3eec61812574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:32.910383 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-e73c32a0-73f6-4eb9-b92c-3eec61812574 None None] MFA Rules not processed for user `ec7412a9a3dc46369fe7b4771e3437de`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:32.911791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e73c32a0-73f6-4eb9-b92c-3eec61812574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:32.962209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1173/2368] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 22:48:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:32.973998 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbac1e3-d2be-4abd-99f1-6faccf1327f5 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:32.975724 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-8cbac1e3-d2be-4abd-99f1-6faccf1327f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:32.983376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:32.989692 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbac1e3-d2be-4abd-99f1-6faccf1327f5 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:32.994656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197df02-9d96-4141-b4ff-999bc42d3b6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:32.994656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197df02-9d96-4141-b4ff-999bc42d3b6f service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:32.994656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197df02-9d96-4141-b4ff-999bc42d3b6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:32.994945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197df02-9d96-4141-b4ff-999bc42d3b6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:33.001717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197df02-9d96-4141-b4ff-999bc42d3b6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:33.003148 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197df02-9d96-4141-b4ff-999bc42d3b6f service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:33.012523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1196/2369] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:48:33.105687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d57e85-6c54-4ce0-a317-dbfdb028b236 None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:33.106085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d57e85-6c54-4ce0-a317-dbfdb028b236 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:33.106378 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d57e85-6c54-4ce0-a317-dbfdb028b236 None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:33.107782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1174/2370] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 22:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:48:33.114951 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458afbb-a80e-41a9-a900-03522968ce46 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:33.115246 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458afbb-a80e-41a9-a900-03522968ce46 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:33.115526 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458afbb-a80e-41a9-a900-03522968ce46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:33.134832 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-a458afbb-a80e-41a9-a900-03522968ce46 None None] MFA Rules not processed for user `159cc8befc3a4d57bbd313832084db62`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:33.136038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a458afbb-a80e-41a9-a900-03522968ce46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:33.145897 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1197/2371] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 22:48:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:33.156344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d57e85-6c54-4ce0-a317-dbfdb028b236 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:33.158056 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-65d57e85-6c54-4ce0-a317-dbfdb028b236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:33.171303 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d57e85-6c54-4ce0-a317-dbfdb028b236 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:33.175368 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db307d6-8de3-41d8-b780-b7d7dd778e2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:33.176759 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db307d6-8de3-41d8-b780-b7d7dd778e2f service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:33.177050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db307d6-8de3-41d8-b780-b7d7dd778e2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:33.177331 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db307d6-8de3-41d8-b780-b7d7dd778e2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:33.187812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db307d6-8de3-41d8-b780-b7d7dd778e2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:33.189278 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db307d6-8de3-41d8-b780-b7d7dd778e2f service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:33.200625 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1175/2372] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:48:43.128483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=8 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:43.195967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=62 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:43.659727 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319b3d0-cfe7-4135-bac4-8c7426a0af5e None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:43.660023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319b3d0-cfe7-4135-bac4-8c7426a0af5e None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:43.660334 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319b3d0-cfe7-4135-bac4-8c7426a0af5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:43.679181 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-6319b3d0-cfe7-4135-bac4-8c7426a0af5e None None] MFA Rules not processed for user `0a7c204a298c42f0bbc32fa9a6ce1f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:43.680392 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6319b3d0-cfe7-4135-bac4-8c7426a0af5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:43.735053 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1198/2373] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 22:48:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:43.747624 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db307d6-8de3-41d8-b780-b7d7dd778e2f service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:43.749147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-8db307d6-8de3-41d8-b780-b7d7dd778e2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:43.819120 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db307d6-8de3-41d8-b780-b7d7dd778e2f service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:43.824021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c5153-22f6-4e57-8f90-60b2c3c47804 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b321586cdb3d4d25904472aaa4adba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:43.824701 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c5153-22f6-4e57-8f90-60b2c3c47804 service swift] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:43.824991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c5153-22f6-4e57-8f90-60b2c3c47804 service swift] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:43.825277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c5153-22f6-4e57-8f90-60b2c3c47804 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:43.833338 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c5153-22f6-4e57-8f90-60b2c3c47804 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:43.834967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c5153-22f6-4e57-8f90-60b2c3c47804 service swift] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:43.843137 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1176/2374] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 22:48:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:48:46.063340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319b3d0-cfe7-4135-bac4-8c7426a0af5e None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:46.065076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6319b3d0-cfe7-4135-bac4-8c7426a0af5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:46.141341 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319b3d0-cfe7-4135-bac4-8c7426a0af5e None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:46.146492 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071695c-00ae-419f-8e07-36c2d7234281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:46.147817 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071695c-00ae-419f-8e07-36c2d7234281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:46.148088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071695c-00ae-419f-8e07-36c2d7234281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:46.148372 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071695c-00ae-419f-8e07-36c2d7234281 admin admin] PATH_INFO: `/v3/users/a71edb6b0adc49358358910b36c5bb13` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:46.149747 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071695c-00ae-419f-8e07-36c2d7234281 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a71edb6b0adc49358358910b36c5bb13)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:46.151751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071695c-00ae-419f-8e07-36c2d7234281 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:46.343921 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6071695c-00ae-419f-8e07-36c2d7234281 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a71edb6b0adc49358358910b36c5bb13'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:46.361070 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6071695c-00ae-419f-8e07-36c2d7234281 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a71edb6b0adc49358358910b36c5bb13'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:46.361961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6071695c-00ae-419f-8e07-36c2d7234281 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a71edb6b0adc49358358910b36c5bb13'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:46.362812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6071695c-00ae-419f-8e07-36c2d7234281 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a71edb6b0adc49358358910b36c5bb13'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:46.375335 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6071695c-00ae-419f-8e07-36c2d7234281 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a71edb6b0adc49358358910b36c5bb13'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:46.465088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1199/2375] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:48:46 2022] DELETE /identity/v3/users/a71edb6b0adc49358358910b36c5bb13 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:46.471626 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c710c-3ea0-4fb5-be0c-4fe8f4ed399e None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:46.472218 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c710c-3ea0-4fb5-be0c-4fe8f4ed399e None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:46.472310 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c710c-3ea0-4fb5-be0c-4fe8f4ed399e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:46.492013 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-3c7c710c-3ea0-4fb5-be0c-4fe8f4ed399e None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:46.493286 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3c7c710c-3ea0-4fb5-be0c-4fe8f4ed399e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:46.555383 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1177/2376] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:48:46.568724 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071695c-00ae-419f-8e07-36c2d7234281 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:46.570490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6071695c-00ae-419f-8e07-36c2d7234281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:46.601686 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:46.634550 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071695c-00ae-419f-8e07-36c2d7234281 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:46.638796 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d260c-350f-452f-8a8c-3b27fc28986f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:46.639550 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d260c-350f-452f-8a8c-3b27fc28986f service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:46.639900 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d260c-350f-452f-8a8c-3b27fc28986f service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:46.640238 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d260c-350f-452f-8a8c-3b27fc28986f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:46.651650 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d260c-350f-452f-8a8c-3b27fc28986f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:46.653537 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d260c-350f-452f-8a8c-3b27fc28986f service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:46.664937 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1200/2377] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:48:47.106722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c710c-3ea0-4fb5-be0c-4fe8f4ed399e None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:47.108136 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3c7c710c-3ea0-4fb5-be0c-4fe8f4ed399e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:47.125141 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c710c-3ea0-4fb5-be0c-4fe8f4ed399e None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:47.129923 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce2d03-6ad6-4fb7-a365-e3f042b1c4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:47.130941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce2d03-6ad6-4fb7-a365-e3f042b1c4d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:47.131228 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce2d03-6ad6-4fb7-a365-e3f042b1c4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:47.131503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce2d03-6ad6-4fb7-a365-e3f042b1c4d7 admin admin] PATH_INFO: `/v3/projects/07cbbb995a5543a1b861033a56afdaa3` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:47.132857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce2d03-6ad6-4fb7-a365-e3f042b1c4d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07cbbb995a5543a1b861033a56afdaa3)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:47.134747 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce2d03-6ad6-4fb7-a365-e3f042b1c4d7 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:47.836901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f0ce2d03-6ad6-4fb7-a365-e3f042b1c4d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07cbbb995a5543a1b861033a56afdaa3', 'request_id': 'req-f0ce2d03-6ad6-4fb7-a365-e3f042b1c4d7'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:47.950265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f0ce2d03-6ad6-4fb7-a365-e3f042b1c4d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07cbbb995a5543a1b861033a56afdaa3', 'request_id': 'req-f0ce2d03-6ad6-4fb7-a365-e3f042b1c4d7'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:47.965967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f0ce2d03-6ad6-4fb7-a365-e3f042b1c4d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07cbbb995a5543a1b861033a56afdaa3', 'request_id': 'req-f0ce2d03-6ad6-4fb7-a365-e3f042b1c4d7'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:47.967800 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f0ce2d03-6ad6-4fb7-a365-e3f042b1c4d7 admin admin] The token cache is being invalidate because project 07cbbb995a5543a1b861033a56afdaa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:48:47.968094 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f0ce2d03-6ad6-4fb7-a365-e3f042b1c4d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:47.968772 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f0ce2d03-6ad6-4fb7-a365-e3f042b1c4d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:47.970317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1178/2378] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:48:47 2022] DELETE /identity/v3/projects/07cbbb995a5543a1b861033a56afdaa3 => generated 0 bytes in 864 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:50.566956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c4baa-99e5-4de0-9bcc-dc6854786dce None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:50.567364 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c4baa-99e5-4de0-9bcc-dc6854786dce None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:50.567551 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c4baa-99e5-4de0-9bcc-dc6854786dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:50.587407 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-ac1c4baa-99e5-4de0-9bcc-dc6854786dce None None] MFA Rules not processed for user `dd98b2a31e4c46599269684782915703`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:50.590202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ac1c4baa-99e5-4de0-9bcc-dc6854786dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:50.660830 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1201/2379] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:48:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:48:57.964648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:49:00.654038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=40 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:49:11.266653 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2665625-912b-476d-ab1d-d58449b5289c None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:49:11.267162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2665625-912b-476d-ab1d-d58449b5289c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:49:11.267254 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2665625-912b-476d-ab1d-d58449b5289c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:49:11.286251 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-e2665625-912b-476d-ab1d-d58449b5289c None None] MFA Rules not processed for user `0a7c204a298c42f0bbc32fa9a6ce1f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:49:11.287446 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e2665625-912b-476d-ab1d-d58449b5289c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:49:11.347287 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1179/2380] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 23 22:49:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:49:11.373774 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c4baa-99e5-4de0-9bcc-dc6854786dce None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:49:11.375415 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ac1c4baa-99e5-4de0-9bcc-dc6854786dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:49:11.461070 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c4baa-99e5-4de0-9bcc-dc6854786dce None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:49:11.466541 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713307a-f29b-458f-a0e9-c84106b5df5f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b321586cdb3d4d25904472aaa4adba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:49:11.467192 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713307a-f29b-458f-a0e9-c84106b5df5f service swift] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:49:11.467479 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713307a-f29b-458f-a0e9-c84106b5df5f service swift] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:49:11.467759 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713307a-f29b-458f-a0e9-c84106b5df5f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:49:11.475739 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713307a-f29b-458f-a0e9-c84106b5df5f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:49:11.478001 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713307a-f29b-458f-a0e9-c84106b5df5f service swift] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:49:11.486552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1202/2381] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 22:49:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:49:21.340054 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=25 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:49:21.482955 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=32 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:49:26.721128 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515d01d-55f5-4bb3-a808-14ab933fd358 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:49:26.721669 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515d01d-55f5-4bb3-a808-14ab933fd358 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:49:26.721767 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515d01d-55f5-4bb3-a808-14ab933fd358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:49:26.740278 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-a515d01d-55f5-4bb3-a808-14ab933fd358 None None] MFA Rules not processed for user `1d7e431402f8433fb7565f69103485ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:49:26.741588 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-a515d01d-55f5-4bb3-a808-14ab933fd358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:49:26.797180 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1180/2382] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:49:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:49:36.790092 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=25 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:49:39.528691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272ee11-7e11-4c77-9a04-f988e831c587 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:49:39.529545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272ee11-7e11-4c77-9a04-f988e831c587 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:49:39.530004 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272ee11-7e11-4c77-9a04-f988e831c587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:49:39.548369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-e272ee11-7e11-4c77-9a04-f988e831c587 None None] MFA Rules not processed for user `675d117397da452faa5221af0906fb12`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:49:39.549678 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e272ee11-7e11-4c77-9a04-f988e831c587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:49:39.609640 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1203/2383] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:49:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:49:49.601747 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=25 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:50:06.958749 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68966035-ca3b-437c-8f05-eb6545aa1c9c None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:06.959288 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68966035-ca3b-437c-8f05-eb6545aa1c9c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:06.959288 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68966035-ca3b-437c-8f05-eb6545aa1c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:06.976419 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-68966035-ca3b-437c-8f05-eb6545aa1c9c None None] MFA Rules not processed for user `5107e87ee4c34ef9ac96fa4a3a899bda`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:50:06.977637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-68966035-ca3b-437c-8f05-eb6545aa1c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:07.035330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1181/2384] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:50:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:50:07.073674 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272ee11-7e11-4c77-9a04-f988e831c587 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:07.075591 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e272ee11-7e11-4c77-9a04-f988e831c587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:07.136174 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272ee11-7e11-4c77-9a04-f988e831c587 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:07.139898 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2c703-c9fa-4c55-9f31-db761f984284 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '983801bb18924fe4b3150693765eb13b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:07.140558 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2c703-c9fa-4c55-9f31-db761f984284 service glance] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:07.140873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2c703-c9fa-4c55-9f31-db761f984284 service glance] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:07.141158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2c703-c9fa-4c55-9f31-db761f984284 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:07.142949 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a8d2c703-c9fa-4c55-9f31-db761f984284 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:07.214222 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2c703-c9fa-4c55-9f31-db761f984284 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:07.215663 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2c703-c9fa-4c55-9f31-db761f984284 service glance] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:07.227090 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1204/2385] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:50:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:50:10.104213 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68966035-ca3b-437c-8f05-eb6545aa1c9c None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:10.106017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-68966035-ca3b-437c-8f05-eb6545aa1c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:10.174123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68966035-ca3b-437c-8f05-eb6545aa1c9c None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:10.178960 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cf10b-69b8-4fab-a063-16fbd9016121 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:10.179823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cf10b-69b8-4fab-a063-16fbd9016121 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:10.180269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cf10b-69b8-4fab-a063-16fbd9016121 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:10.180760 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cf10b-69b8-4fab-a063-16fbd9016121 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:10.182798 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e21cf10b-69b8-4fab-a063-16fbd9016121 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:10.201641 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21cf10b-69b8-4fab-a063-16fbd9016121 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:10.203288 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21cf10b-69b8-4fab-a063-16fbd9016121 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:10.215716 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1182/2386] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:50:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:50:13.370559 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2c703-c9fa-4c55-9f31-db761f984284 service glance] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:13.372788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a8d2c703-c9fa-4c55-9f31-db761f984284 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:13.438499 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2c703-c9fa-4c55-9f31-db761f984284 service glance] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:13.443610 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a10ce9-4a40-4b6d-8968-d1f085b1c722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:13.444480 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a10ce9-4a40-4b6d-8968-d1f085b1c722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:13.444811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a10ce9-4a40-4b6d-8968-d1f085b1c722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:13.445095 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a10ce9-4a40-4b6d-8968-d1f085b1c722 admin admin] PATH_INFO: `/v3/users/dd98b2a31e4c46599269684782915703` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:13.446292 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a10ce9-4a40-4b6d-8968-d1f085b1c722 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd98b2a31e4c46599269684782915703)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:13.448224 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a10ce9-4a40-4b6d-8968-d1f085b1c722 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:13.598209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:50:13.600700 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-27a10ce9-4a40-4b6d-8968-d1f085b1c722 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd98b2a31e4c46599269684782915703'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:50:13.612360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-27a10ce9-4a40-4b6d-8968-d1f085b1c722 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd98b2a31e4c46599269684782915703'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:50:13.613180 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-27a10ce9-4a40-4b6d-8968-d1f085b1c722 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd98b2a31e4c46599269684782915703'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:50:13.613798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-27a10ce9-4a40-4b6d-8968-d1f085b1c722 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd98b2a31e4c46599269684782915703'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:50:13.620610 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-27a10ce9-4a40-4b6d-8968-d1f085b1c722 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd98b2a31e4c46599269684782915703'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:50:13.688894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1205/2387] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:50:13 2022] DELETE /identity/v3/users/dd98b2a31e4c46599269684782915703 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:50:14.016724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cf10b-69b8-4fab-a063-16fbd9016121 service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:14.029905 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cf10b-69b8-4fab-a063-16fbd9016121 service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:14.088958 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd6bd5-8344-4237-a7cc-7b75a53dcf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:14.089823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd6bd5-8344-4237-a7cc-7b75a53dcf22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:14.090104 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd6bd5-8344-4237-a7cc-7b75a53dcf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:14.090377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd6bd5-8344-4237-a7cc-7b75a53dcf22 admin admin] PATH_INFO: `/v3/projects/cb1c26e7fcbd4c0c9bb4a68e54569e3a` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:14.091560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fd6bd5-8344-4237-a7cc-7b75a53dcf22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb1c26e7fcbd4c0c9bb4a68e54569e3a)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:14.093800 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fd6bd5-8344-4237-a7cc-7b75a53dcf22 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:14.229338 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-82fd6bd5-8344-4237-a7cc-7b75a53dcf22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb1c26e7fcbd4c0c9bb4a68e54569e3a', 'request_id': 'req-82fd6bd5-8344-4237-a7cc-7b75a53dcf22'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:50:14.231131 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:50:14.312790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-82fd6bd5-8344-4237-a7cc-7b75a53dcf22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb1c26e7fcbd4c0c9bb4a68e54569e3a', 'request_id': 'req-82fd6bd5-8344-4237-a7cc-7b75a53dcf22'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:50:14.321080 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-82fd6bd5-8344-4237-a7cc-7b75a53dcf22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb1c26e7fcbd4c0c9bb4a68e54569e3a', 'request_id': 'req-82fd6bd5-8344-4237-a7cc-7b75a53dcf22'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:50:14.336554 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-82fd6bd5-8344-4237-a7cc-7b75a53dcf22 admin admin] The token cache is being invalidate because project cb1c26e7fcbd4c0c9bb4a68e54569e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:50:14.336921 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-82fd6bd5-8344-4237-a7cc-7b75a53dcf22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:50:14.337555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-82fd6bd5-8344-4237-a7cc-7b75a53dcf22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:50:14.339191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1183/2388] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:50:14 2022] DELETE /identity/v3/projects/cb1c26e7fcbd4c0c9bb4a68e54569e3a => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:50:14.344371 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68828ee-bc81-4613-b154-cd62495067cd None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:14.344696 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68828ee-bc81-4613-b154-cd62495067cd None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:14.345003 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68828ee-bc81-4613-b154-cd62495067cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:14.364198 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-f68828ee-bc81-4613-b154-cd62495067cd None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:50:14.365459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-f68828ee-bc81-4613-b154-cd62495067cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:14.429355 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1206/2389] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:50:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:50:14.436865 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458387a-ab4a-4a62-8718-c54d8d3feb5c None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:14.437154 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458387a-ab4a-4a62-8718-c54d8d3feb5c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:14.437436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458387a-ab4a-4a62-8718-c54d8d3feb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:14.456201 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-4458387a-ab4a-4a62-8718-c54d8d3feb5c None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:50:14.457507 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-4458387a-ab4a-4a62-8718-c54d8d3feb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:14.465646 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1184/2390] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:50:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:50:14.469897 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68828ee-bc81-4613-b154-cd62495067cd None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:14.480050 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68828ee-bc81-4613-b154-cd62495067cd None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:14.484688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386030a5-bee5-4398-b6ed-395c3d8c1168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:14.485347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386030a5-bee5-4398-b6ed-395c3d8c1168 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:14.485631 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386030a5-bee5-4398-b6ed-395c3d8c1168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:14.485908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386030a5-bee5-4398-b6ed-395c3d8c1168 admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:14.486913 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386030a5-bee5-4398-b6ed-395c3d8c1168 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:50:14.487229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386030a5-bee5-4398-b6ed-395c3d8c1168 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:14.489574 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386030a5-bee5-4398-b6ed-395c3d8c1168 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:14.501401 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1207/2391] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:50:14.506093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458387a-ab4a-4a62-8718-c54d8d3feb5c None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:14.515336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458387a-ab4a-4a62-8718-c54d8d3feb5c None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:14.520022 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b95f6-2dec-4151-bfd3-fcd7da0b3738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:14.520720 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b95f6-2dec-4151-bfd3-fcd7da0b3738 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:14.521011 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b95f6-2dec-4151-bfd3-fcd7da0b3738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:14.521296 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b95f6-2dec-4151-bfd3-fcd7da0b3738 admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:14.522024 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b95f6-2dec-4151-bfd3-fcd7da0b3738 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:14.523934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b95f6-2dec-4151-bfd3-fcd7da0b3738 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:14.609677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1185/2392] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:50:14 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:50:14.614155 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386030a5-bee5-4398-b6ed-395c3d8c1168 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:14.624431 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386030a5-bee5-4398-b6ed-395c3d8c1168 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:14.679978 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fedbd-cc9b-4d40-991b-c914ca726bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:14.680558 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fedbd-cc9b-4d40-991b-c914ca726bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:14.680868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fedbd-cc9b-4d40-991b-c914ca726bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:14.681145 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fedbd-cc9b-4d40-991b-c914ca726bc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:14.681852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40fedbd-cc9b-4d40-991b-c914ca726bc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:14.683746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40fedbd-cc9b-4d40-991b-c914ca726bc9 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:14.751226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1208/2393] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:50:14 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:50:14.755417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b95f6-2dec-4151-bfd3-fcd7da0b3738 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:14.765676 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b95f6-2dec-4151-bfd3-fcd7da0b3738 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:14.770971 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b288d8-87dc-4469-9dcc-6b463a4126c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:14.771629 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b288d8-87dc-4469-9dcc-6b463a4126c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:14.771918 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b288d8-87dc-4469-9dcc-6b463a4126c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:14.772200 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b288d8-87dc-4469-9dcc-6b463a4126c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:14.772963 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b288d8-87dc-4469-9dcc-6b463a4126c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:14.774565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b288d8-87dc-4469-9dcc-6b463a4126c1 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:14.784874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1186/2394] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:50:14.789318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fedbd-cc9b-4d40-991b-c914ca726bc9 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:14.799211 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fedbd-cc9b-4d40-991b-c914ca726bc9 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:14.803945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b62787-32e4-433b-8212-9f0f748e07ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:14.804619 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b62787-32e4-433b-8212-9f0f748e07ee admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:14.804913 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b62787-32e4-433b-8212-9f0f748e07ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:14.805193 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b62787-32e4-433b-8212-9f0f748e07ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:14.805917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b62787-32e4-433b-8212-9f0f748e07ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:14.807503 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b62787-32e4-433b-8212-9f0f748e07ee admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:14.818382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1209/2395] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:50:14.822267 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b288d8-87dc-4469-9dcc-6b463a4126c1 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:14.832026 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b288d8-87dc-4469-9dcc-6b463a4126c1 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:14.837132 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf221d-bbfa-4cf1-9aaa-992d3a33e600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:14.837997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf221d-bbfa-4cf1-9aaa-992d3a33e600 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:14.838284 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf221d-bbfa-4cf1-9aaa-992d3a33e600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:14.838563 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf221d-bbfa-4cf1-9aaa-992d3a33e600 admin admin] PATH_INFO: `/v3/projects/e3f62d8ed71b424f8414e9c3eaa681aa/users/e6a2de8d5a514a73aa4e136b7afee9c2/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:14.857409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf221d-bbfa-4cf1-9aaa-992d3a33e600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3f62d8ed71b424f8414e9c3eaa681aa, user_id=e6a2de8d5a514a73aa4e136b7afee9c2, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:14.860079 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf221d-bbfa-4cf1-9aaa-992d3a33e600 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:14.953346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1187/2396] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:50:14 2022] PUT /identity/v3/projects/e3f62d8ed71b424f8414e9c3eaa681aa/users/e6a2de8d5a514a73aa4e136b7afee9c2/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:50:14.959114 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47b292-e5fa-4e40-8310-60d9108c8a55 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:14.959407 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47b292-e5fa-4e40-8310-60d9108c8a55 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:14.959686 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47b292-e5fa-4e40-8310-60d9108c8a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:14.977941 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-fc47b292-e5fa-4e40-8310-60d9108c8a55 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:50:14.979205 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-fc47b292-e5fa-4e40-8310-60d9108c8a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:15.001465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:50:15.043783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1210/2397] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:50:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:50:15.053755 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf221d-bbfa-4cf1-9aaa-992d3a33e600 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:15.055598 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-5acf221d-bbfa-4cf1-9aaa-992d3a33e600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:15.114900 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf221d-bbfa-4cf1-9aaa-992d3a33e600 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:15.118535 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70a685-6093-4271-81f6-feeca2d0f46e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:15.120967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70a685-6093-4271-81f6-feeca2d0f46e service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:15.120967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70a685-6093-4271-81f6-feeca2d0f46e service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:15.120967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70a685-6093-4271-81f6-feeca2d0f46e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:15.128774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70a685-6093-4271-81f6-feeca2d0f46e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:15.130835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70a685-6093-4271-81f6-feeca2d0f46e service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:15.140203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1188/2398] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:50:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:50:17.317629 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47b292-e5fa-4e40-8310-60d9108c8a55 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:17.320004 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-fc47b292-e5fa-4e40-8310-60d9108c8a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:17.465116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47b292-e5fa-4e40-8310-60d9108c8a55 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:17.478560 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506b795-5872-4e7a-9b20-563811aea98e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:17.479464 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506b795-5872-4e7a-9b20-563811aea98e service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:17.479650 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506b795-5872-4e7a-9b20-563811aea98e service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:17.480023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506b795-5872-4e7a-9b20-563811aea98e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:17.485686 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-9506b795-5872-4e7a-9b20-563811aea98e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:17.624112 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506b795-5872-4e7a-9b20-563811aea98e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:17.625933 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506b795-5872-4e7a-9b20-563811aea98e service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:17.636634 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1211/2399] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:50:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:50:20.820729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1b017-a4ae-4798-bf1b-f405cb0f1bab None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:20.821608 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1b017-a4ae-4798-bf1b-f405cb0f1bab None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:20.822055 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1b017-a4ae-4798-bf1b-f405cb0f1bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:20.857008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-7de1b017-a4ae-4798-bf1b-f405cb0f1bab None None] MFA Rules not processed for user `84a5fb262284400591cf29dd13771ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:50:20.859905 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-7de1b017-a4ae-4798-bf1b-f405cb0f1bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:20.971078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1189/2400] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:50:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:50:21.089169 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a23e7-21b9-4cc8-aa5f-05252c78fce5 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:21.089699 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a23e7-21b9-4cc8-aa5f-05252c78fce5 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:21.089904 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a23e7-21b9-4cc8-aa5f-05252c78fce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:21.158348 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-9b2a23e7-21b9-4cc8-aa5f-05252c78fce5 None None] MFA Rules not processed for user `e6a2de8d5a514a73aa4e136b7afee9c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:50:21.166165 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-9b2a23e7-21b9-4cc8-aa5f-05252c78fce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:21.254721 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:50:21.270903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1212/2401] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:50:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:50:21.277535 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1b017-a4ae-4798-bf1b-f405cb0f1bab None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:21.302100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1b017-a4ae-4798-bf1b-f405cb0f1bab None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:21.307176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c31430-425a-4000-9857-ce6927f8274d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:21.307857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c31430-425a-4000-9857-ce6927f8274d admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:21.308366 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c31430-425a-4000-9857-ce6927f8274d admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:21.308836 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c31430-425a-4000-9857-ce6927f8274d admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:21.309696 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c31430-425a-4000-9857-ce6927f8274d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:21.311884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c31430-425a-4000-9857-ce6927f8274d admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:21.485674 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:50:21.504906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1190/2402] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:50:21 2022] POST /identity/v3/projects => generated 385 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:50:21.506887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a23e7-21b9-4cc8-aa5f-05252c78fce5 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:21.523467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a23e7-21b9-4cc8-aa5f-05252c78fce5 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:21.610993 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12a0f5-e233-42a6-afc1-d3b24058940e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:21.612374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12a0f5-e233-42a6-afc1-d3b24058940e admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:21.612885 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12a0f5-e233-42a6-afc1-d3b24058940e admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:21.613586 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12a0f5-e233-42a6-afc1-d3b24058940e admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:21.614603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12a0f5-e233-42a6-afc1-d3b24058940e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:21.617785 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12a0f5-e233-42a6-afc1-d3b24058940e admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:21.745106 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c31430-425a-4000-9857-ce6927f8274d admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:21.748665 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1213/2403] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:50:21 2022] POST /identity/v3/users => generated 299 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:50:21.769297 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c31430-425a-4000-9857-ce6927f8274d admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:21.775766 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923c616-e774-4cbd-855c-63ae5a9510c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:21.776455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923c616-e774-4cbd-855c-63ae5a9510c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:21.776455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923c616-e774-4cbd-855c-63ae5a9510c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:21.776873 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923c616-e774-4cbd-855c-63ae5a9510c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:21.777693 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a923c616-e774-4cbd-855c-63ae5a9510c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:21.780148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a923c616-e774-4cbd-855c-63ae5a9510c9 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:21.796879 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1191/2404] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:50:21.801909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12a0f5-e233-42a6-afc1-d3b24058940e admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:21.816826 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12a0f5-e233-42a6-afc1-d3b24058940e admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:21.822685 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb48b9-581e-4706-91d4-8ba540ef2428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:21.823664 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb48b9-581e-4706-91d4-8ba540ef2428 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:21.824041 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb48b9-581e-4706-91d4-8ba540ef2428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:21.824396 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb48b9-581e-4706-91d4-8ba540ef2428 admin admin] PATH_INFO: `/v3/projects/17d5c759351445b6aeaf4f31685f321b/users/498c2d8bc3f1481099c60293525cb33f/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:21.849431 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb48b9-581e-4706-91d4-8ba540ef2428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17d5c759351445b6aeaf4f31685f321b, user_id=498c2d8bc3f1481099c60293525cb33f, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:21.853261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb48b9-581e-4706-91d4-8ba540ef2428 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:21.940767 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1214/2405] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:50:21 2022] PUT /identity/v3/projects/17d5c759351445b6aeaf4f31685f321b/users/498c2d8bc3f1481099c60293525cb33f/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:50:21.946432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923c616-e774-4cbd-855c-63ae5a9510c9 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:21.961200 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923c616-e774-4cbd-855c-63ae5a9510c9 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:22.053626 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50661a89-45c5-44b9-bc39-62b7d4f8baa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:22.054594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50661a89-45c5-44b9-bc39-62b7d4f8baa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:22.054979 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50661a89-45c5-44b9-bc39-62b7d4f8baa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:22.055514 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50661a89-45c5-44b9-bc39-62b7d4f8baa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:22.056358 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50661a89-45c5-44b9-bc39-62b7d4f8baa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:22.058241 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50661a89-45c5-44b9-bc39-62b7d4f8baa1 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:22.071346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1192/2406] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:50:22.075451 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb48b9-581e-4706-91d4-8ba540ef2428 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:22.101866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb48b9-581e-4706-91d4-8ba540ef2428 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:22.101866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b03c6e-4fcf-4291-826b-45cd0c7cfbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:22.101866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b03c6e-4fcf-4291-826b-45cd0c7cfbe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:22.101866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b03c6e-4fcf-4291-826b-45cd0c7cfbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:22.102707 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b03c6e-4fcf-4291-826b-45cd0c7cfbe7 admin admin] PATH_INFO: `/v3/projects/17d5c759351445b6aeaf4f31685f321b/users/498c2d8bc3f1481099c60293525cb33f/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:22.102707 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b03c6e-4fcf-4291-826b-45cd0c7cfbe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17d5c759351445b6aeaf4f31685f321b, user_id=498c2d8bc3f1481099c60293525cb33f, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:22.102707 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b03c6e-4fcf-4291-826b-45cd0c7cfbe7 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:22.109402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1215/2407] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:50:22 2022] PUT /identity/v3/projects/17d5c759351445b6aeaf4f31685f321b/users/498c2d8bc3f1481099c60293525cb33f/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:50:26.543362 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50661a89-45c5-44b9-bc39-62b7d4f8baa1 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:26.546188 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-50661a89-45c5-44b9-bc39-62b7d4f8baa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:26.694354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50661a89-45c5-44b9-bc39-62b7d4f8baa1 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:26.802764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f399b0-9d4c-4c0b-a9ba-8ad4fd416686 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:26.803746 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f399b0-9d4c-4c0b-a9ba-8ad4fd416686 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:26.804041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f399b0-9d4c-4c0b-a9ba-8ad4fd416686 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:26.804338 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f399b0-9d4c-4c0b-a9ba-8ad4fd416686 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:26.806678 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-f4f399b0-9d4c-4c0b-a9ba-8ad4fd416686 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:26.835749 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b03c6e-4fcf-4291-826b-45cd0c7cfbe7 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:26.838552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-b8b03c6e-4fcf-4291-826b-45cd0c7cfbe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:26.930177 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:50:26.943489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f399b0-9d4c-4c0b-a9ba-8ad4fd416686 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:26.947206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f399b0-9d4c-4c0b-a9ba-8ad4fd416686 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:26.974417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1193/2408] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:50:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:50:27.029605 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b03c6e-4fcf-4291-826b-45cd0c7cfbe7 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:27.037153 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ae31a-3e5b-4b50-b1b2-844359843df6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:27.038165 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ae31a-3e5b-4b50-b1b2-844359843df6 service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:27.038453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ae31a-3e5b-4b50-b1b2-844359843df6 service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:27.039019 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ae31a-3e5b-4b50-b1b2-844359843df6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:27.054545 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42ae31a-3e5b-4b50-b1b2-844359843df6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:27.057758 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42ae31a-3e5b-4b50-b1b2-844359843df6 service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:27.076441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1216/2409] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:50:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:50:28.673987 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f399b0-9d4c-4c0b-a9ba-8ad4fd416686 service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:28.699523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f399b0-9d4c-4c0b-a9ba-8ad4fd416686 service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:28.708775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9d183-626a-49ea-a22f-a01c65090f99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:28.709449 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9d183-626a-49ea-a22f-a01c65090f99 service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:28.709735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9d183-626a-49ea-a22f-a01c65090f99 service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:28.710012 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9d183-626a-49ea-a22f-a01c65090f99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:28.716387 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-68f9d183-626a-49ea-a22f-a01c65090f99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:28.868509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9d183-626a-49ea-a22f-a01c65090f99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:28.868509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9d183-626a-49ea-a22f-a01c65090f99 service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:28.882451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1194/2410] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:50:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:50:29.497079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3cd00-85af-4046-8061-65500b9f7e89 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:29.497615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3cd00-85af-4046-8061-65500b9f7e89 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:29.497615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3cd00-85af-4046-8061-65500b9f7e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:29.537365 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-1ac3cd00-85af-4046-8061-65500b9f7e89 None None] MFA Rules not processed for user `498c2d8bc3f1481099c60293525cb33f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:50:29.553462 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-1ac3cd00-85af-4046-8061-65500b9f7e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:29.561764 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:50:29.620712 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1217/2411] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:50:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:50:29.637524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9d183-626a-49ea-a22f-a01c65090f99 service nova] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:29.639063 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-68f9d183-626a-49ea-a22f-a01c65090f99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:29.698968 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9d183-626a-49ea-a22f-a01c65090f99 service nova] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:29.702865 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71feb8d-252e-4797-b3e8-12acfcc78597 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '983801bb18924fe4b3150693765eb13b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:29.703511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71feb8d-252e-4797-b3e8-12acfcc78597 service glance] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:29.703794 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71feb8d-252e-4797-b3e8-12acfcc78597 service glance] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:29.704071 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71feb8d-252e-4797-b3e8-12acfcc78597 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:29.772849 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71feb8d-252e-4797-b3e8-12acfcc78597 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:29.774311 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71feb8d-252e-4797-b3e8-12acfcc78597 service glance] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:29.785667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1195/2412] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:50:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:50:30.012733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2db02f-d134-4393-b0c2-47d47108d168 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:30.013038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2db02f-d134-4393-b0c2-47d47108d168 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:30.013321 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2db02f-d134-4393-b0c2-47d47108d168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:30.031762 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-0b2db02f-d134-4393-b0c2-47d47108d168 None None] MFA Rules not processed for user `0a7c204a298c42f0bbc32fa9a6ce1f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:50:30.033035 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-0b2db02f-d134-4393-b0c2-47d47108d168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:30.097644 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1218/2413] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 22:50:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:50:30.265539 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71feb8d-252e-4797-b3e8-12acfcc78597 service glance] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:30.267024 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-a71feb8d-252e-4797-b3e8-12acfcc78597 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:30.319716 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:50:30.339254 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71feb8d-252e-4797-b3e8-12acfcc78597 service glance] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:30.344754 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c34ff-d70b-4039-afa7-f85e4fb04503 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b321586cdb3d4d25904472aaa4adba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:30.345300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c34ff-d70b-4039-afa7-f85e4fb04503 service swift] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:30.345588 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c34ff-d70b-4039-afa7-f85e4fb04503 service swift] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:30.345869 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c34ff-d70b-4039-afa7-f85e4fb04503 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:30.355981 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c34ff-d70b-4039-afa7-f85e4fb04503 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:30.361560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c34ff-d70b-4039-afa7-f85e4fb04503 service swift] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:30.367702 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1196/2414] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 22:50:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 22:50:40.090330 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=45 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:50:40.413252 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=10 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:50:41.803475 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd20693-e3f4-4909-a2ba-6fbd646ccb72 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:41.804024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd20693-e3f4-4909-a2ba-6fbd646ccb72 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:41.804199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd20693-e3f4-4909-a2ba-6fbd646ccb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:41.823163 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-ccd20693-e3f4-4909-a2ba-6fbd646ccb72 None None] MFA Rules not processed for user `e6a2de8d5a514a73aa4e136b7afee9c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:50:41.824861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ccd20693-e3f4-4909-a2ba-6fbd646ccb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:41.833148 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1219/2415] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:50:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:50:42.110434 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c34ff-d70b-4039-afa7-f85e4fb04503 service swift] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:42.165225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c34ff-d70b-4039-afa7-f85e4fb04503 service swift] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:42.168783 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93812be-71f9-4e1b-9467-b07135e6af4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:42.169433 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93812be-71f9-4e1b-9467-b07135e6af4e service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:42.169712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93812be-71f9-4e1b-9467-b07135e6af4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:42.169993 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93812be-71f9-4e1b-9467-b07135e6af4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:42.171773 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-a93812be-71f9-4e1b-9467-b07135e6af4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:42.181464 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93812be-71f9-4e1b-9467-b07135e6af4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:42.182895 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93812be-71f9-4e1b-9467-b07135e6af4e service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:42.191739 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1197/2416] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:50:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:50:42.592468 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd20693-e3f4-4909-a2ba-6fbd646ccb72 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:42.594613 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ccd20693-e3f4-4909-a2ba-6fbd646ccb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:42.672247 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd20693-e3f4-4909-a2ba-6fbd646ccb72 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:42.677837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc7398-5bab-4224-9096-18d2a30a3c2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ac44b3f0224e56a15993a2f29c1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:42.679492 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc7398-5bab-4224-9096-18d2a30a3c2a service placement] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:42.679492 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc7398-5bab-4224-9096-18d2a30a3c2a service placement] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:42.679492 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc7398-5bab-4224-9096-18d2a30a3c2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:42.681305 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-82dc7398-5bab-4224-9096-18d2a30a3c2a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:42.693789 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc7398-5bab-4224-9096-18d2a30a3c2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:42.695399 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc7398-5bab-4224-9096-18d2a30a3c2a service placement] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:42.705363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1220/2417] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:50:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:50:46.805757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94e857-a091-442f-bed3-53c69d2a2480 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:46.806270 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94e857-a091-442f-bed3-53c69d2a2480 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:46.806392 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94e857-a091-442f-bed3-53c69d2a2480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:46.827994 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-3b94e857-a091-442f-bed3-53c69d2a2480 None None] MFA Rules not processed for user `0a7c204a298c42f0bbc32fa9a6ce1f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:50:46.829887 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3b94e857-a091-442f-bed3-53c69d2a2480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:46.841086 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1198/2418] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 22:50:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:50:46.862175 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc7398-5bab-4224-9096-18d2a30a3c2a service placement] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:46.874002 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc7398-5bab-4224-9096-18d2a30a3c2a service placement] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:46.879130 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126c556-9d24-45a9-bfe8-77321788a345 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b321586cdb3d4d25904472aaa4adba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:46.879780 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126c556-9d24-45a9-bfe8-77321788a345 service swift] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:46.880065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126c556-9d24-45a9-bfe8-77321788a345 service swift] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:46.880340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126c556-9d24-45a9-bfe8-77321788a345 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:46.889124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8126c556-9d24-45a9-bfe8-77321788a345 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:46.891089 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8126c556-9d24-45a9-bfe8-77321788a345 service swift] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:46.904299 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1221/2419] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 22:50:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:50:56.821863 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=29 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:50:56.900091 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=41 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:50:59.991395 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94e857-a091-442f-bed3-53c69d2a2480 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:59.992893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3b94e857-a091-442f-bed3-53c69d2a2480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:59.994498 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126c556-9d24-45a9-bfe8-77321788a345 service swift] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:59.995965 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-8126c556-9d24-45a9-bfe8-77321788a345 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:00.005277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94e857-a091-442f-bed3-53c69d2a2480 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:00.008455 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126c556-9d24-45a9-bfe8-77321788a345 service swift] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:00.008950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae25a8-1bca-4869-b313-3540355bd6b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:00.009593 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae25a8-1bca-4869-b313-3540355bd6b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:00.009878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae25a8-1bca-4869-b313-3540355bd6b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:00.010355 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae25a8-1bca-4869-b313-3540355bd6b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:00.012134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-5bae25a8-1bca-4869-b313-3540355bd6b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:00.012277 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92811815-1eac-4a9c-83fa-c8298a7ba379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:00.012828 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92811815-1eac-4a9c-83fa-c8298a7ba379 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:00.013119 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92811815-1eac-4a9c-83fa-c8298a7ba379 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:00.013405 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92811815-1eac-4a9c-83fa-c8298a7ba379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:00.015164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-92811815-1eac-4a9c-83fa-c8298a7ba379 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:00.025529 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae25a8-1bca-4869-b313-3540355bd6b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:00.026236 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92811815-1eac-4a9c-83fa-c8298a7ba379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:00.027842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92811815-1eac-4a9c-83fa-c8298a7ba379 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:00.027962 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae25a8-1bca-4869-b313-3540355bd6b0 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:00.037313 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1199/2420] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:50:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:51:00.037815 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1222/2421] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:50:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:51:00.712374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae25a8-1bca-4869-b313-3540355bd6b0 service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:51:00.725917 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae25a8-1bca-4869-b313-3540355bd6b0 service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:00.731675 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed5fc7-d2f7-4e28-a25a-abbf999a50ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:00.732462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed5fc7-d2f7-4e28-a25a-abbf999a50ef service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:00.732840 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed5fc7-d2f7-4e28-a25a-abbf999a50ef service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:00.733212 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed5fc7-d2f7-4e28-a25a-abbf999a50ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:00.735523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c7ed5fc7-d2f7-4e28-a25a-abbf999a50ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:00.749688 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed5fc7-d2f7-4e28-a25a-abbf999a50ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:00.751352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed5fc7-d2f7-4e28-a25a-abbf999a50ef service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:00.764262 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1200/2422] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:51:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:51:02.311447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92811815-1eac-4a9c-83fa-c8298a7ba379 service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:51:02.322200 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92811815-1eac-4a9c-83fa-c8298a7ba379 service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:02.325774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed5fc7-d2f7-4e28-a25a-abbf999a50ef service nova] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:51:02.326698 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3d7c1-4a5d-4679-9449-a5598b120a8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:02.327406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3d7c1-4a5d-4679-9449-a5598b120a8c service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:02.327688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3d7c1-4a5d-4679-9449-a5598b120a8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:02.328494 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3d7c1-4a5d-4679-9449-a5598b120a8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:02.330439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-1cf3d7c1-4a5d-4679-9449-a5598b120a8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:02.337993 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed5fc7-d2f7-4e28-a25a-abbf999a50ef service nova] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:02.344698 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1037d4-331a-4c08-b3ed-b00bf898c049 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:02.345348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1037d4-331a-4c08-b3ed-b00bf898c049 service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:02.345630 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1037d4-331a-4c08-b3ed-b00bf898c049 service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:02.345928 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1037d4-331a-4c08-b3ed-b00bf898c049 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:02.347774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-6d1037d4-331a-4c08-b3ed-b00bf898c049 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:02.360805 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1037d4-331a-4c08-b3ed-b00bf898c049 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:02.362886 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1037d4-331a-4c08-b3ed-b00bf898c049 service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:02.375312 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1201/2423] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:51:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:51:02.409645 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3d7c1-4a5d-4679-9449-a5598b120a8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:02.411594 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3d7c1-4a5d-4679-9449-a5598b120a8c service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:02.426596 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1223/2424] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:51:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:51:07.035734 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fb9d0-b629-4da0-bfd7-66aea97dcc3a None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:07.036329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fb9d0-b629-4da0-bfd7-66aea97dcc3a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:07.036644 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fb9d0-b629-4da0-bfd7-66aea97dcc3a None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:07.043695 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1202/2425] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 22:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:51:07.056423 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dcdc6-cd9b-4821-b88f-2358d7ed6d67 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:07.056934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dcdc6-cd9b-4821-b88f-2358d7ed6d67 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:07.057046 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dcdc6-cd9b-4821-b88f-2358d7ed6d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:07.105060 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-8c4dcdc6-cd9b-4821-b88f-2358d7ed6d67 None None] MFA Rules not processed for user `d3ac44b3f0224e56a15993a2f29c1e29`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:51:07.106277 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-8c4dcdc6-cd9b-4821-b88f-2358d7ed6d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:07.115400 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1224/2426] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 22:51:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:51:07.135090 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fb9d0-b629-4da0-bfd7-66aea97dcc3a None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:51:07.137039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-b30fb9d0-b629-4da0-bfd7-66aea97dcc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:07.159206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fb9d0-b629-4da0-bfd7-66aea97dcc3a None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:07.164998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118ee69-3a40-4462-824a-968cd4573571 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ac44b3f0224e56a15993a2f29c1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:07.165869 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118ee69-3a40-4462-824a-968cd4573571 service placement] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:07.166155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118ee69-3a40-4462-824a-968cd4573571 service placement] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:07.166516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118ee69-3a40-4462-824a-968cd4573571 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:07.175695 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118ee69-3a40-4462-824a-968cd4573571 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:07.177364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118ee69-3a40-4462-824a-968cd4573571 service placement] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:07.187210 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1203/2427] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:51:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:51:07.251820 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4dcdc6-cd9b-4821-b88f-2358d7ed6d67 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:51:07.253515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-8c4dcdc6-cd9b-4821-b88f-2358d7ed6d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:07.270126 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4dcdc6-cd9b-4821-b88f-2358d7ed6d67 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:07.275023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f93f3-c4ca-4a72-9e0a-a76427b700ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:07.276793 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f93f3-c4ca-4a72-9e0a-a76427b700ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:07.276793 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f93f3-c4ca-4a72-9e0a-a76427b700ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:07.276793 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f93f3-c4ca-4a72-9e0a-a76427b700ce admin admin] PATH_INFO: `/v3/users/675d117397da452faa5221af0906fb12` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:07.278008 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f93f3-c4ca-4a72-9e0a-a76427b700ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=675d117397da452faa5221af0906fb12)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:07.279935 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f93f3-c4ca-4a72-9e0a-a76427b700ce admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:07.479467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-f54f93f3-c4ca-4a72-9e0a-a76427b700ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '675d117397da452faa5221af0906fb12'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:51:07.491774 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-f54f93f3-c4ca-4a72-9e0a-a76427b700ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '675d117397da452faa5221af0906fb12'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:51:07.494755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-f54f93f3-c4ca-4a72-9e0a-a76427b700ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '675d117397da452faa5221af0906fb12'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:51:07.494755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-f54f93f3-c4ca-4a72-9e0a-a76427b700ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '675d117397da452faa5221af0906fb12'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:51:07.500129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-f54f93f3-c4ca-4a72-9e0a-a76427b700ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '675d117397da452faa5221af0906fb12'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:51:07.567337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1225/2428] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:51:07 2022] DELETE /identity/v3/users/675d117397da452faa5221af0906fb12 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:51:07.780544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118ee69-3a40-4462-824a-968cd4573571 service placement] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:51:07.799683 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118ee69-3a40-4462-824a-968cd4573571 service placement] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:07.863377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d1d78-5e72-4129-b08b-80a56fd7c741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:07.864197 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d1d78-5e72-4129-b08b-80a56fd7c741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:07.864484 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d1d78-5e72-4129-b08b-80a56fd7c741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:07.864791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d1d78-5e72-4129-b08b-80a56fd7c741 admin admin] PATH_INFO: `/v3/projects/19ed20d55acb41fcb18006919476bf80` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:07.865997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d1d78-5e72-4129-b08b-80a56fd7c741 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19ed20d55acb41fcb18006919476bf80)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:07.867970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d1d78-5e72-4129-b08b-80a56fd7c741 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:08.006731 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ce8d1d78-5e72-4129-b08b-80a56fd7c741 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19ed20d55acb41fcb18006919476bf80', 'request_id': 'req-ce8d1d78-5e72-4129-b08b-80a56fd7c741'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:51:08.114057 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ce8d1d78-5e72-4129-b08b-80a56fd7c741 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19ed20d55acb41fcb18006919476bf80', 'request_id': 'req-ce8d1d78-5e72-4129-b08b-80a56fd7c741'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:51:08.122548 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ce8d1d78-5e72-4129-b08b-80a56fd7c741 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19ed20d55acb41fcb18006919476bf80', 'request_id': 'req-ce8d1d78-5e72-4129-b08b-80a56fd7c741'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:51:08.134618 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ce8d1d78-5e72-4129-b08b-80a56fd7c741 admin admin] The token cache is being invalidate because project 19ed20d55acb41fcb18006919476bf80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:51:08.137815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ce8d1d78-5e72-4129-b08b-80a56fd7c741 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:51:08.137815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ce8d1d78-5e72-4129-b08b-80a56fd7c741 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:51:08.137815 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1204/2429] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:51:07 2022] DELETE /identity/v3/projects/19ed20d55acb41fcb18006919476bf80 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:51:08.143481 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cad7de-5dea-4b35-bfa0-cc5042d29042 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:08.144030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cad7de-5dea-4b35-bfa0-cc5042d29042 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:08.144244 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cad7de-5dea-4b35-bfa0-cc5042d29042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:08.169088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-e9cad7de-5dea-4b35-bfa0-cc5042d29042 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:51:08.170652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e9cad7de-5dea-4b35-bfa0-cc5042d29042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:08.205693 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:51:08.241090 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d1d78-5e72-4129-b08b-80a56fd7c741 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:51:08.246398 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ce8d1d78-5e72-4129-b08b-80a56fd7c741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:08.262783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1226/2430] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:51:08.271718 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc050d2-8869-45f0-b159-9a7da08e43d4 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:08.272217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc050d2-8869-45f0-b159-9a7da08e43d4 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:08.272504 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc050d2-8869-45f0-b159-9a7da08e43d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:08.291725 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-0bc050d2-8869-45f0-b159-9a7da08e43d4 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:51:08.293229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-0bc050d2-8869-45f0-b159-9a7da08e43d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:08.301904 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1227/2431] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:51:08.306580 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc050d2-8869-45f0-b159-9a7da08e43d4 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:51:08.308208 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:51:08.318640 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc050d2-8869-45f0-b159-9a7da08e43d4 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:08.323875 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d1532-5d03-47c1-8a60-18edd9aba32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:08.324763 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d1532-5d03-47c1-8a60-18edd9aba32b admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:08.325207 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d1532-5d03-47c1-8a60-18edd9aba32b admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:08.325669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d1532-5d03-47c1-8a60-18edd9aba32b admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:08.326706 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d1532-5d03-47c1-8a60-18edd9aba32b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:51:08.327183 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d1532-5d03-47c1-8a60-18edd9aba32b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:08.328831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d1532-5d03-47c1-8a60-18edd9aba32b admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:08.343958 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1228/2432] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:51:08.348902 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d1532-5d03-47c1-8a60-18edd9aba32b admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:51:08.358411 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d1d78-5e72-4129-b08b-80a56fd7c741 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:08.361567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d1532-5d03-47c1-8a60-18edd9aba32b admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:08.366515 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b3b83-0303-4594-9a30-06f8b5b66479 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:08.367342 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b3b83-0303-4594-9a30-06f8b5b66479 service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:08.367804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b3b83-0303-4594-9a30-06f8b5b66479 service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:08.368423 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b3b83-0303-4594-9a30-06f8b5b66479 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:08.368854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4981372-7dec-4ec0-b4d1-7739c1d65419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:08.369581 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4981372-7dec-4ec0-b4d1-7739c1d65419 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:08.369877 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4981372-7dec-4ec0-b4d1-7739c1d65419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:08.370556 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4981372-7dec-4ec0-b4d1-7739c1d65419 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:08.371375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-a70b3b83-0303-4594-9a30-06f8b5b66479 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:08.371497 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4981372-7dec-4ec0-b4d1-7739c1d65419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:08.373706 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4981372-7dec-4ec0-b4d1-7739c1d65419 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:08.389822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b3b83-0303-4594-9a30-06f8b5b66479 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:08.392407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b3b83-0303-4594-9a30-06f8b5b66479 service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:08.406104 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1205/2433] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:51:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:51:08.489556 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1229/2434] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:51:08 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:51:08.493488 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b3b83-0303-4594-9a30-06f8b5b66479 service nova] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:51:08.506368 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b3b83-0303-4594-9a30-06f8b5b66479 service nova] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:08.577440 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a56e8-f6c3-49b8-bcb3-c560821febee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:08.578377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a56e8-f6c3-49b8-bcb3-c560821febee admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:08.578748 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a56e8-f6c3-49b8-bcb3-c560821febee admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:08.579105 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a56e8-f6c3-49b8-bcb3-c560821febee admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:08.579990 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a56e8-f6c3-49b8-bcb3-c560821febee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:08.582215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a56e8-f6c3-49b8-bcb3-c560821febee admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:08.654460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1206/2435] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:51:08 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:51:08.658892 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4981372-7dec-4ec0-b4d1-7739c1d65419 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:51:08.669182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4981372-7dec-4ec0-b4d1-7739c1d65419 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:08.674291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7b202-c589-4503-8f25-41e7db895010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:08.674977 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7b202-c589-4503-8f25-41e7db895010 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:08.675265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7b202-c589-4503-8f25-41e7db895010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:08.676019 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7b202-c589-4503-8f25-41e7db895010 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:08.676846 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7b202-c589-4503-8f25-41e7db895010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:08.678456 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7b202-c589-4503-8f25-41e7db895010 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:08.688987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1230/2436] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:51:08.693974 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a56e8-f6c3-49b8-bcb3-c560821febee admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:51:08.707121 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a56e8-f6c3-49b8-bcb3-c560821febee admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:08.712965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac27ef8-960e-4f93-a98a-1e67154ac625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:08.713643 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac27ef8-960e-4f93-a98a-1e67154ac625 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:08.713929 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac27ef8-960e-4f93-a98a-1e67154ac625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:08.714210 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac27ef8-960e-4f93-a98a-1e67154ac625 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:08.715127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac27ef8-960e-4f93-a98a-1e67154ac625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:08.716841 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac27ef8-960e-4f93-a98a-1e67154ac625 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:08.728093 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1207/2437] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:51:08.732513 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7b202-c589-4503-8f25-41e7db895010 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:51:08.743472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7b202-c589-4503-8f25-41e7db895010 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:08.749962 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542affb3-6465-401a-ae48-6fd7cfc5451e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:08.756766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542affb3-6465-401a-ae48-6fd7cfc5451e admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:08.756766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542affb3-6465-401a-ae48-6fd7cfc5451e admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:08.756766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542affb3-6465-401a-ae48-6fd7cfc5451e admin admin] PATH_INFO: `/v3/projects/73d61a582ec24f99aaf018e55f1f049b/users/92ce6158277146228cb13194e9d7b04b/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:08.766971 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542affb3-6465-401a-ae48-6fd7cfc5451e admin admin] RBAC: Authorizing `identity:create_grant(project_id=73d61a582ec24f99aaf018e55f1f049b, user_id=92ce6158277146228cb13194e9d7b04b, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:08.770205 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542affb3-6465-401a-ae48-6fd7cfc5451e admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:08.840817 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1231/2438] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:51:08 2022] PUT /identity/v3/projects/73d61a582ec24f99aaf018e55f1f049b/users/92ce6158277146228cb13194e9d7b04b/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:51:08.846435 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a1386-1522-43b0-8ee9-1da2cf2fa131 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:08.846752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a1386-1522-43b0-8ee9-1da2cf2fa131 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:08.847013 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a1386-1522-43b0-8ee9-1da2cf2fa131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:08.865425 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-006a1386-1522-43b0-8ee9-1da2cf2fa131 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:51:08.866627 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-006a1386-1522-43b0-8ee9-1da2cf2fa131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:08.916722 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a2d38-13ee-491a-8fe2-c744f90cba0e None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:08.917015 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a2d38-13ee-491a-8fe2-c744f90cba0e None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:08.917298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a2d38-13ee-491a-8fe2-c744f90cba0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:08.926518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1208/2439] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:51:08.935333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-062a2d38-13ee-491a-8fe2-c744f90cba0e None None] MFA Rules not processed for user `1d7e431402f8433fb7565f69103485ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:51:08.935954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a1386-1522-43b0-8ee9-1da2cf2fa131 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:51:08.936632 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-062a2d38-13ee-491a-8fe2-c744f90cba0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:08.940299 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-006a1386-1522-43b0-8ee9-1da2cf2fa131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:08.993204 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a1386-1522-43b0-8ee9-1da2cf2fa131 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:08.994385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1232/2440] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:51:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:51:08.999011 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07bc5f-7d7e-478b-aebc-d986d1144522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:08.999656 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07bc5f-7d7e-478b-aebc-d986d1144522 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:08.999941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07bc5f-7d7e-478b-aebc-d986d1144522 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:09.000221 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07bc5f-7d7e-478b-aebc-d986d1144522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:09.010473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec07bc5f-7d7e-478b-aebc-d986d1144522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:09.011905 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec07bc5f-7d7e-478b-aebc-d986d1144522 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:09.022264 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1209/2441] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:51:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:51:09.168392 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:51:09.301338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a2d38-13ee-491a-8fe2-c744f90cba0e None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:51:09.302961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-062a2d38-13ee-491a-8fe2-c744f90cba0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:09.316431 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a2d38-13ee-491a-8fe2-c744f90cba0e None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:09.320622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671617d7-8cb5-4f5d-8051-1cb01d2a7b3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:09.321286 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671617d7-8cb5-4f5d-8051-1cb01d2a7b3b service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:09.321597 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671617d7-8cb5-4f5d-8051-1cb01d2a7b3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:09.321875 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671617d7-8cb5-4f5d-8051-1cb01d2a7b3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:09.323801 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-671617d7-8cb5-4f5d-8051-1cb01d2a7b3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:09.335334 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671617d7-8cb5-4f5d-8051-1cb01d2a7b3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:09.339050 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671617d7-8cb5-4f5d-8051-1cb01d2a7b3b service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:09.347057 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1233/2442] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:51:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:51:16.322333 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09dd64-7417-471d-9b48-947ca2cf2cdb None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:16.322878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09dd64-7417-471d-9b48-947ca2cf2cdb None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:16.322878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09dd64-7417-471d-9b48-947ca2cf2cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:16.353125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-7f09dd64-7417-471d-9b48-947ca2cf2cdb None None] MFA Rules not processed for user `92ce6158277146228cb13194e9d7b04b`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:51:16.354218 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-7f09dd64-7417-471d-9b48-947ca2cf2cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:16.455470 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1210/2443] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:51:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:51:16.469123 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671617d7-8cb5-4f5d-8051-1cb01d2a7b3b service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:51:16.471075 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-671617d7-8cb5-4f5d-8051-1cb01d2a7b3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:16.526693 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:51:16.553038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671617d7-8cb5-4f5d-8051-1cb01d2a7b3b service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:16.558687 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62618b7c-d9c4-4685-a790-a8036b9bd457 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:16.559293 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62618b7c-d9c4-4685-a790-a8036b9bd457 service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:16.559584 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62618b7c-d9c4-4685-a790-a8036b9bd457 service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:16.559869 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62618b7c-d9c4-4685-a790-a8036b9bd457 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:16.569011 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62618b7c-d9c4-4685-a790-a8036b9bd457 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:16.570665 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62618b7c-d9c4-4685-a790-a8036b9bd457 service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:16.580383 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1234/2444] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:51:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:51:19.113528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1007c-07f7-4d8e-ba1b-d1331fd178a8 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:19.114110 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1007c-07f7-4d8e-ba1b-d1331fd178a8 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:19.114110 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1007c-07f7-4d8e-ba1b-d1331fd178a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:19.133809 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-23a1007c-07f7-4d8e-ba1b-d1331fd178a8 None None] MFA Rules not processed for user `92ce6158277146228cb13194e9d7b04b`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:51:19.135283 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-23a1007c-07f7-4d8e-ba1b-d1331fd178a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:19.149466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1211/2445] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:51:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:51:22.613033 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62618b7c-d9c4-4685-a790-a8036b9bd457 service nova] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:51:22.625936 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62618b7c-d9c4-4685-a790-a8036b9bd457 service nova] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:22.631950 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4ed09-439e-45d1-a4ce-7fb2da78a3fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:22.633172 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4ed09-439e-45d1-a4ce-7fb2da78a3fd service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:22.640716 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4ed09-439e-45d1-a4ce-7fb2da78a3fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:22.640716 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4ed09-439e-45d1-a4ce-7fb2da78a3fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:22.640716 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-37c4ed09-439e-45d1-a4ce-7fb2da78a3fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:22.733177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4ed09-439e-45d1-a4ce-7fb2da78a3fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:22.734971 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4ed09-439e-45d1-a4ce-7fb2da78a3fd service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:22.747529 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1235/2446] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:51:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:51:25.684952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e55f3-ec6f-4031-b44d-0ac350069a79 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:25.686086 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e55f3-ec6f-4031-b44d-0ac350069a79 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:25.686579 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e55f3-ec6f-4031-b44d-0ac350069a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:25.708410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-2f7e55f3-ec6f-4031-b44d-0ac350069a79 None None] MFA Rules not processed for user `0a7c204a298c42f0bbc32fa9a6ce1f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:51:25.710060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-2f7e55f3-ec6f-4031-b44d-0ac350069a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:25.798757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1212/2447] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 22:51:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:51:25.807311 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4ed09-439e-45d1-a4ce-7fb2da78a3fd service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:51:25.809718 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-37c4ed09-439e-45d1-a4ce-7fb2da78a3fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:25.927606 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4ed09-439e-45d1-a4ce-7fb2da78a3fd service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:25.944120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e58cf8-e5b2-4bab-ad54-f2602a37d6d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b321586cdb3d4d25904472aaa4adba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:25.945123 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e58cf8-e5b2-4bab-ad54-f2602a37d6d6 service swift] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:25.945410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e58cf8-e5b2-4bab-ad54-f2602a37d6d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:25.945689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e58cf8-e5b2-4bab-ad54-f2602a37d6d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:25.962316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e58cf8-e5b2-4bab-ad54-f2602a37d6d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:25.964971 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e58cf8-e5b2-4bab-ad54-f2602a37d6d6 service swift] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:25.977704 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1236/2448] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 22:51:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:51:35.785200 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=67 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:51:35.971813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=74 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:51:47.008395 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029a471-a8bd-41fb-bdfa-6d9a998729df None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:47.008945 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029a471-a8bd-41fb-bdfa-6d9a998729df None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:47.009033 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029a471-a8bd-41fb-bdfa-6d9a998729df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:47.028461 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-b029a471-a8bd-41fb-bdfa-6d9a998729df None None] MFA Rules not processed for user `0a7c204a298c42f0bbc32fa9a6ce1f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:51:47.029694 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-b029a471-a8bd-41fb-bdfa-6d9a998729df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:51:47.037757 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1213/2449] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 22:51:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:51:47.049723 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e58cf8-e5b2-4bab-ad54-f2602a37d6d6 service swift] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:51:47.060221 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e58cf8-e5b2-4bab-ad54-f2602a37d6d6 service swift] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:51:47.065628 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8d2fd-fce1-46fd-9630-76a4658c87d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b321586cdb3d4d25904472aaa4adba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:51:47.066263 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8d2fd-fce1-46fd-9630-76a4658c87d6 service swift] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:51:47.066549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8d2fd-fce1-46fd-9630-76a4658c87d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:51:47.066825 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8d2fd-fce1-46fd-9630-76a4658c87d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:51:47.074930 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8d2fd-fce1-46fd-9630-76a4658c87d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:51:47.076510 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8d2fd-fce1-46fd-9630-76a4658c87d6 service swift] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:51:47.085252 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1237/2450] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 22:51:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 22:51:57.028289 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:51:57.082050 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=6 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:52:04.330132 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295dfb9-c377-4698-a322-722fe4ebaf03 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:04.330600 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295dfb9-c377-4698-a322-722fe4ebaf03 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:04.331543 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295dfb9-c377-4698-a322-722fe4ebaf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:04.348409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-c295dfb9-c377-4698-a322-722fe4ebaf03 None None] MFA Rules not processed for user `0a7c204a298c42f0bbc32fa9a6ce1f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:52:04.349627 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c295dfb9-c377-4698-a322-722fe4ebaf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:04.356535 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1214/2451] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 23 22:52:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:52:04.368957 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8d2fd-fce1-46fd-9630-76a4658c87d6 service swift] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:04.379852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8d2fd-fce1-46fd-9630-76a4658c87d6 service swift] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:04.384945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a200714-3857-4857-a52a-a75e62f19914 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b321586cdb3d4d25904472aaa4adba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:04.385578 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a200714-3857-4857-a52a-a75e62f19914 service swift] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:04.385860 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a200714-3857-4857-a52a-a75e62f19914 service swift] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:04.386135 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a200714-3857-4857-a52a-a75e62f19914 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:04.394912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a200714-3857-4857-a52a-a75e62f19914 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:04.396482 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a200714-3857-4857-a52a-a75e62f19914 service swift] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:04.404448 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1238/2452] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 22:52:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:52:14.343360 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:52:14.401216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=6 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:52:22.157026 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295dfb9-c377-4698-a322-722fe4ebaf03 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:22.158745 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c295dfb9-c377-4698-a322-722fe4ebaf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:22.215902 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295dfb9-c377-4698-a322-722fe4ebaf03 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:22.219654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724dddb-614c-4d86-84fc-f898e5ac0c9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:22.220320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724dddb-614c-4d86-84fc-f898e5ac0c9a service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:22.221631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724dddb-614c-4d86-84fc-f898e5ac0c9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:22.221925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724dddb-614c-4d86-84fc-f898e5ac0c9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:22.223781 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-1724dddb-614c-4d86-84fc-f898e5ac0c9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:22.282670 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724dddb-614c-4d86-84fc-f898e5ac0c9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:22.284970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724dddb-614c-4d86-84fc-f898e5ac0c9a service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:22.294135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1215/2453] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:52:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:52:30.833956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bca94e-f6f1-4ded-96c9-936c83f952f8 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:30.834498 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bca94e-f6f1-4ded-96c9-936c83f952f8 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:30.834498 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bca94e-f6f1-4ded-96c9-936c83f952f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:30.851902 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-38bca94e-f6f1-4ded-96c9-936c83f952f8 None None] MFA Rules not processed for user `84a5fb262284400591cf29dd13771ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:52:30.853132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-38bca94e-f6f1-4ded-96c9-936c83f952f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:30.860239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1239/2454] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:52:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:52:32.290141 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=48 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:52:38.834849 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724dddb-614c-4d86-84fc-f898e5ac0c9a service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:38.844024 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724dddb-614c-4d86-84fc-f898e5ac0c9a service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:38.847897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd926de-21e8-4feb-8ffd-a2f883b974e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:38.848539 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd926de-21e8-4feb-8ffd-a2f883b974e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:38.848855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd926de-21e8-4feb-8ffd-a2f883b974e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:38.849132 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd926de-21e8-4feb-8ffd-a2f883b974e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:38.850945 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ecd926de-21e8-4feb-8ffd-a2f883b974e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:38.862951 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd926de-21e8-4feb-8ffd-a2f883b974e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:38.864370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd926de-21e8-4feb-8ffd-a2f883b974e0 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:38.873995 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1216/2455] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:52:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:52:40.846479 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:52:42.548024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bca94e-f6f1-4ded-96c9-936c83f952f8 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:42.549682 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-38bca94e-f6f1-4ded-96c9-936c83f952f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:42.565936 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bca94e-f6f1-4ded-96c9-936c83f952f8 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:42.569431 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b17e4-2899-4d30-9024-bebc9bd03a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:42.570316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b17e4-2899-4d30-9024-bebc9bd03a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:42.570788 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b17e4-2899-4d30-9024-bebc9bd03a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:42.571230 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b17e4-2899-4d30-9024-bebc9bd03a1d admin admin] PATH_INFO: `/v3/users/1d7e431402f8433fb7565f69103485ce` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:42.572832 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b17e4-2899-4d30-9024-bebc9bd03a1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d7e431402f8433fb7565f69103485ce)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:42.574860 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b17e4-2899-4d30-9024-bebc9bd03a1d admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:42.715987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-683b17e4-2899-4d30-9024-bebc9bd03a1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d7e431402f8433fb7565f69103485ce'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:42.729326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-683b17e4-2899-4d30-9024-bebc9bd03a1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d7e431402f8433fb7565f69103485ce'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:42.730056 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-683b17e4-2899-4d30-9024-bebc9bd03a1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d7e431402f8433fb7565f69103485ce'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:42.730673 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-683b17e4-2899-4d30-9024-bebc9bd03a1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d7e431402f8433fb7565f69103485ce'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:42.738250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-683b17e4-2899-4d30-9024-bebc9bd03a1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d7e431402f8433fb7565f69103485ce'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:42.800132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1240/2456] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:52:42 2022] DELETE /identity/v3/users/1d7e431402f8433fb7565f69103485ce => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:52:43.222309 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd926de-21e8-4feb-8ffd-a2f883b974e0 service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:43.233470 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd926de-21e8-4feb-8ffd-a2f883b974e0 service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:43.292408 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0261825-ab91-4b2b-8ea1-4eca24bc3dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:43.292408 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0261825-ab91-4b2b-8ea1-4eca24bc3dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:43.292774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0261825-ab91-4b2b-8ea1-4eca24bc3dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:43.293055 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0261825-ab91-4b2b-8ea1-4eca24bc3dd0 admin admin] PATH_INFO: `/v3/projects/18ed82f19aa2407dac94c2dc67a3fa9c` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:43.294262 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0261825-ab91-4b2b-8ea1-4eca24bc3dd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18ed82f19aa2407dac94c2dc67a3fa9c)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:43.296237 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0261825-ab91-4b2b-8ea1-4eca24bc3dd0 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:43.396098 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-b0261825-ab91-4b2b-8ea1-4eca24bc3dd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18ed82f19aa2407dac94c2dc67a3fa9c', 'request_id': 'req-b0261825-ab91-4b2b-8ea1-4eca24bc3dd0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:43.443956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-b0261825-ab91-4b2b-8ea1-4eca24bc3dd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18ed82f19aa2407dac94c2dc67a3fa9c', 'request_id': 'req-b0261825-ab91-4b2b-8ea1-4eca24bc3dd0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:43.450391 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-b0261825-ab91-4b2b-8ea1-4eca24bc3dd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18ed82f19aa2407dac94c2dc67a3fa9c', 'request_id': 'req-b0261825-ab91-4b2b-8ea1-4eca24bc3dd0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:43.461335 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-b0261825-ab91-4b2b-8ea1-4eca24bc3dd0 admin admin] The token cache is being invalidate because project 18ed82f19aa2407dac94c2dc67a3fa9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:52:43.461624 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-b0261825-ab91-4b2b-8ea1-4eca24bc3dd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:43.462263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-b0261825-ab91-4b2b-8ea1-4eca24bc3dd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:43.463803 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1217/2457] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:52:43 2022] DELETE /identity/v3/projects/18ed82f19aa2407dac94c2dc67a3fa9c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:52:43.468889 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52908b45-8bd7-456a-b71a-7059c2b64430 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:43.469184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52908b45-8bd7-456a-b71a-7059c2b64430 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:43.469470 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52908b45-8bd7-456a-b71a-7059c2b64430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:43.488147 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-52908b45-8bd7-456a-b71a-7059c2b64430 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:52:43.489337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-52908b45-8bd7-456a-b71a-7059c2b64430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:43.548868 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1241/2458] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:52:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:52:43.556960 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da6e33-c17c-4636-bae9-76cb3996b9d8 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:43.557262 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da6e33-c17c-4636-bae9-76cb3996b9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:43.557555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da6e33-c17c-4636-bae9-76cb3996b9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:43.576297 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-c6da6e33-c17c-4636-bae9-76cb3996b9d8 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:52:43.577557 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c6da6e33-c17c-4636-bae9-76cb3996b9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:43.586224 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1218/2459] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:52:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:52:43.590336 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52908b45-8bd7-456a-b71a-7059c2b64430 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:43.600046 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52908b45-8bd7-456a-b71a-7059c2b64430 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:43.604600 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc2fd8-d0f0-476c-8ede-4d0b50edfb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:43.605269 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc2fd8-d0f0-476c-8ede-4d0b50edfb39 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:43.605556 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc2fd8-d0f0-476c-8ede-4d0b50edfb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:43.606184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc2fd8-d0f0-476c-8ede-4d0b50edfb39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:43.607060 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc2fd8-d0f0-476c-8ede-4d0b50edfb39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:52:43.607374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc2fd8-d0f0-476c-8ede-4d0b50edfb39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:43.608994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc2fd8-d0f0-476c-8ede-4d0b50edfb39 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:43.621249 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1242/2460] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:52:43.625437 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da6e33-c17c-4636-bae9-76cb3996b9d8 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:43.635339 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da6e33-c17c-4636-bae9-76cb3996b9d8 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:43.639966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6a480-a8c5-43b3-9f29-d3bb1f2307b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:43.640646 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6a480-a8c5-43b3-9f29-d3bb1f2307b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:43.640940 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6a480-a8c5-43b3-9f29-d3bb1f2307b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:43.641225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6a480-a8c5-43b3-9f29-d3bb1f2307b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:43.641915 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6a480-a8c5-43b3-9f29-d3bb1f2307b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:43.644600 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6a480-a8c5-43b3-9f29-d3bb1f2307b5 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:43.741035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1219/2461] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:52:43 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:52:43.744822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc2fd8-d0f0-476c-8ede-4d0b50edfb39 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:43.754949 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc2fd8-d0f0-476c-8ede-4d0b50edfb39 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:43.811730 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7221719-03bf-40f1-9a4e-bb8d72190eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:43.812398 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7221719-03bf-40f1-9a4e-bb8d72190eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:43.812733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7221719-03bf-40f1-9a4e-bb8d72190eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:43.813020 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7221719-03bf-40f1-9a4e-bb8d72190eb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:43.813715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7221719-03bf-40f1-9a4e-bb8d72190eb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:43.815718 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7221719-03bf-40f1-9a4e-bb8d72190eb9 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:43.822318 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:52:43.911148 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1243/2462] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:52:43 2022] POST /identity/v3/users => generated 283 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:52:43.915275 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6a480-a8c5-43b3-9f29-d3bb1f2307b5 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:43.925663 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6a480-a8c5-43b3-9f29-d3bb1f2307b5 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:43.930240 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5374f-8d8d-4e34-8a13-83169f53da8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:43.930903 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5374f-8d8d-4e34-8a13-83169f53da8e admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:43.931192 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5374f-8d8d-4e34-8a13-83169f53da8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:43.931475 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5374f-8d8d-4e34-8a13-83169f53da8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:43.932206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd5374f-8d8d-4e34-8a13-83169f53da8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:43.933860 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd5374f-8d8d-4e34-8a13-83169f53da8e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:43.943807 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1220/2463] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:52:43.947632 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7221719-03bf-40f1-9a4e-bb8d72190eb9 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:43.958076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7221719-03bf-40f1-9a4e-bb8d72190eb9 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:43.963261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8c807-b4e1-4d20-a914-cd8c6e63daa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:43.963914 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8c807-b4e1-4d20-a914-cd8c6e63daa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:43.964197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8c807-b4e1-4d20-a914-cd8c6e63daa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:43.964473 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8c807-b4e1-4d20-a914-cd8c6e63daa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:43.965304 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8c807-b4e1-4d20-a914-cd8c6e63daa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:43.966892 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8c807-b4e1-4d20-a914-cd8c6e63daa0 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:43.982209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1244/2464] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:52:43.983158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5374f-8d8d-4e34-8a13-83169f53da8e admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:43.993522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5374f-8d8d-4e34-8a13-83169f53da8e admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:43.998408 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521f1fc-6a4b-4acf-9407-5a331a3a93f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:43.999306 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521f1fc-6a4b-4acf-9407-5a331a3a93f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:43.999597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521f1fc-6a4b-4acf-9407-5a331a3a93f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:43.999878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521f1fc-6a4b-4acf-9407-5a331a3a93f9 admin admin] PATH_INFO: `/v3/projects/d881f1edc0404ed6827f5c8b62141837/users/f9a5da2e42c7490c979d628aa25d6866/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:44.015499 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9521f1fc-6a4b-4acf-9407-5a331a3a93f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d881f1edc0404ed6827f5c8b62141837, user_id=f9a5da2e42c7490c979d628aa25d6866, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:44.018857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9521f1fc-6a4b-4acf-9407-5a331a3a93f9 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:44.090201 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1221/2465] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:52:43 2022] PUT /identity/v3/projects/d881f1edc0404ed6827f5c8b62141837/users/f9a5da2e42c7490c979d628aa25d6866/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:52:44.100565 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3a0e3-2375-4896-93e8-0e4543eaa23d None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:44.100885 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3a0e3-2375-4896-93e8-0e4543eaa23d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:44.101162 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3a0e3-2375-4896-93e8-0e4543eaa23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:44.130281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-9dd3a0e3-2375-4896-93e8-0e4543eaa23d None None] MFA Rules not processed for user `f9a5da2e42c7490c979d628aa25d6866`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:52:44.131568 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-9dd3a0e3-2375-4896-93e8-0e4543eaa23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:44.197931 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1245/2466] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:52:44 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:52:44.202383 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521f1fc-6a4b-4acf-9407-5a331a3a93f9 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:44.208904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:52:44.274090 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521f1fc-6a4b-4acf-9407-5a331a3a93f9 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:44.280764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebb550-dc2a-4fe8-bf17-89e9c75e7118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:44.280764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebb550-dc2a-4fe8-bf17-89e9c75e7118 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:44.280764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebb550-dc2a-4fe8-bf17-89e9c75e7118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:44.280764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebb550-dc2a-4fe8-bf17-89e9c75e7118 admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:44.281320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ebb550-dc2a-4fe8-bf17-89e9c75e7118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:44.283030 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ebb550-dc2a-4fe8-bf17-89e9c75e7118 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:44.374603 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1222/2467] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:52:44 2022] POST /identity/v3/projects => generated 365 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:52:44.379402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3a0e3-2375-4896-93e8-0e4543eaa23d None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:44.390751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3a0e3-2375-4896-93e8-0e4543eaa23d None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:44.445719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f08464-f744-495d-a361-45e5e7817780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:44.446383 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f08464-f744-495d-a361-45e5e7817780 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:44.446671 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f08464-f744-495d-a361-45e5e7817780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:44.446982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f08464-f744-495d-a361-45e5e7817780 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:44.447730 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f08464-f744-495d-a361-45e5e7817780 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:44.449684 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f08464-f744-495d-a361-45e5e7817780 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:44.524646 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1246/2468] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:52:44 2022] POST /identity/v3/users => generated 289 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:52:44.528475 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebb550-dc2a-4fe8-bf17-89e9c75e7118 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:44.538902 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebb550-dc2a-4fe8-bf17-89e9c75e7118 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:44.543498 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f6905-0e68-4472-ba61-45db3c20a6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:44.544154 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f6905-0e68-4472-ba61-45db3c20a6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:44.544438 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f6905-0e68-4472-ba61-45db3c20a6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:44.544756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f6905-0e68-4472-ba61-45db3c20a6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:44.545480 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f6905-0e68-4472-ba61-45db3c20a6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:44.547086 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f6905-0e68-4472-ba61-45db3c20a6d0 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:44.558035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1223/2469] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:52:44.568041 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f08464-f744-495d-a361-45e5e7817780 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:44.578124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f08464-f744-495d-a361-45e5e7817780 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:44.582878 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf86e4e-2c15-4789-b8c3-21fc2af0952b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:44.583757 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf86e4e-2c15-4789-b8c3-21fc2af0952b admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:44.584076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf86e4e-2c15-4789-b8c3-21fc2af0952b admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:44.584361 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf86e4e-2c15-4789-b8c3-21fc2af0952b admin admin] PATH_INFO: `/v3/projects/78f8d7e5f62741e48543f0cf89b6371d/users/24e2d97943aa47458db1f273a81c7b37/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:44.599363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf86e4e-2c15-4789-b8c3-21fc2af0952b admin admin] RBAC: Authorizing `identity:create_grant(project_id=78f8d7e5f62741e48543f0cf89b6371d, user_id=24e2d97943aa47458db1f273a81c7b37, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:44.602068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf86e4e-2c15-4789-b8c3-21fc2af0952b admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:44.661071 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1247/2470] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:52:44 2022] PUT /identity/v3/projects/78f8d7e5f62741e48543f0cf89b6371d/users/24e2d97943aa47458db1f273a81c7b37/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:52:44.665057 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f6905-0e68-4472-ba61-45db3c20a6d0 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:44.675385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f6905-0e68-4472-ba61-45db3c20a6d0 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:44.730971 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d4c8c3-f23c-4a1a-9f07-13c848d3f950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:44.731632 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4c8c3-f23c-4a1a-9f07-13c848d3f950 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:44.731911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4c8c3-f23c-4a1a-9f07-13c848d3f950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:44.732190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4c8c3-f23c-4a1a-9f07-13c848d3f950 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:44.732944 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d4c8c3-f23c-4a1a-9f07-13c848d3f950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:44.734524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d4c8c3-f23c-4a1a-9f07-13c848d3f950 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:44.744462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1224/2471] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:52:44.748621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf86e4e-2c15-4789-b8c3-21fc2af0952b admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:44.759475 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf86e4e-2c15-4789-b8c3-21fc2af0952b admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:44.764039 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90410f-195e-4256-ac4a-19aa97a2e49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:44.764945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90410f-195e-4256-ac4a-19aa97a2e49a admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:44.765231 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90410f-195e-4256-ac4a-19aa97a2e49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:44.765510 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90410f-195e-4256-ac4a-19aa97a2e49a admin admin] PATH_INFO: `/v3/projects/78f8d7e5f62741e48543f0cf89b6371d/users/24e2d97943aa47458db1f273a81c7b37/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:44.767495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90410f-195e-4256-ac4a-19aa97a2e49a admin admin] RBAC: Authorizing `identity:create_grant(project_id=78f8d7e5f62741e48543f0cf89b6371d, user_id=24e2d97943aa47458db1f273a81c7b37, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:44.770203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90410f-195e-4256-ac4a-19aa97a2e49a admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:44.780338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1248/2472] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:52:44 2022] PUT /identity/v3/projects/78f8d7e5f62741e48543f0cf89b6371d/users/24e2d97943aa47458db1f273a81c7b37/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:52:44.791544 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae901911-a9e5-4148-adbf-5b2d57aef4c3 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:44.791925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae901911-a9e5-4148-adbf-5b2d57aef4c3 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:44.792211 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae901911-a9e5-4148-adbf-5b2d57aef4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:44.819263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-ae901911-a9e5-4148-adbf-5b2d57aef4c3 None None] MFA Rules not processed for user `24e2d97943aa47458db1f273a81c7b37`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:52:44.820459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ae901911-a9e5-4148-adbf-5b2d57aef4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:44.883075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:52:44.886624 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1225/2473] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:52:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:52:44.897879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90410f-195e-4256-ac4a-19aa97a2e49a admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:44.899661 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ed90410f-195e-4256-ac4a-19aa97a2e49a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:44.923809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:52:44.957626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90410f-195e-4256-ac4a-19aa97a2e49a admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:44.961202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268364db-91f9-4ff0-a9b9-ac1124b18df3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:44.961835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268364db-91f9-4ff0-a9b9-ac1124b18df3 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:44.962118 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268364db-91f9-4ff0-a9b9-ac1124b18df3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:44.962392 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268364db-91f9-4ff0-a9b9-ac1124b18df3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:44.971613 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268364db-91f9-4ff0-a9b9-ac1124b18df3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:44.973357 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268364db-91f9-4ff0-a9b9-ac1124b18df3 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:44.983599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1249/2474] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:52:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:52:45.195179 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae901911-a9e5-4148-adbf-5b2d57aef4c3 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:45.204651 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae901911-a9e5-4148-adbf-5b2d57aef4c3 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:45.208746 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5ce2f-ccb3-47ee-b281-5ee4bff3edf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:45.209393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5ce2f-ccb3-47ee-b281-5ee4bff3edf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:45.209679 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5ce2f-ccb3-47ee-b281-5ee4bff3edf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:45.209958 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5ce2f-ccb3-47ee-b281-5ee4bff3edf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:45.269122 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5ce2f-ccb3-47ee-b281-5ee4bff3edf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:45.270599 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5ce2f-ccb3-47ee-b281-5ee4bff3edf1 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:45.280403 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1226/2475] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:52:45 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:52:48.861487 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268364db-91f9-4ff0-a9b9-ac1124b18df3 service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:48.936813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268364db-91f9-4ff0-a9b9-ac1124b18df3 service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:48.943076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710b77b-0fb7-4f3b-8ec5-2af9744c76b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:48.943273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710b77b-0fb7-4f3b-8ec5-2af9744c76b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:48.943438 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710b77b-0fb7-4f3b-8ec5-2af9744c76b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:48.943717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710b77b-0fb7-4f3b-8ec5-2af9744c76b9 admin admin] PATH_INFO: `/v3/users/f9a5da2e42c7490c979d628aa25d6866` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:48.944955 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710b77b-0fb7-4f3b-8ec5-2af9744c76b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9a5da2e42c7490c979d628aa25d6866)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:48.946898 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710b77b-0fb7-4f3b-8ec5-2af9744c76b9 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:49.165334 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c710b77b-0fb7-4f3b-8ec5-2af9744c76b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9a5da2e42c7490c979d628aa25d6866'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:49.177531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c710b77b-0fb7-4f3b-8ec5-2af9744c76b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9a5da2e42c7490c979d628aa25d6866'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:49.178301 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c710b77b-0fb7-4f3b-8ec5-2af9744c76b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9a5da2e42c7490c979d628aa25d6866'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:49.178939 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c710b77b-0fb7-4f3b-8ec5-2af9744c76b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9a5da2e42c7490c979d628aa25d6866'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:49.185844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c710b77b-0fb7-4f3b-8ec5-2af9744c76b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9a5da2e42c7490c979d628aa25d6866'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:49.296204 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1250/2476] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:52:48 2022] DELETE /identity/v3/users/f9a5da2e42c7490c979d628aa25d6866 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:52:49.300921 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c404b-e8e9-4895-b44a-b60737f09edf None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:49.301442 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c404b-e8e9-4895-b44a-b60737f09edf None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:49.301554 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c404b-e8e9-4895-b44a-b60737f09edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:49.320902 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-805c404b-e8e9-4895-b44a-b60737f09edf None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:52:49.322107 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-805c404b-e8e9-4895-b44a-b60737f09edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:49.384793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1227/2477] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:52:49.396928 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710b77b-0fb7-4f3b-8ec5-2af9744c76b9 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:49.398357 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-c710b77b-0fb7-4f3b-8ec5-2af9744c76b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:49.444827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:52:49.456862 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710b77b-0fb7-4f3b-8ec5-2af9744c76b9 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:49.460663 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1ad92-d612-48db-a0a2-2cf98e6d29bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:49.461313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1ad92-d612-48db-a0a2-2cf98e6d29bd service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:49.461599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1ad92-d612-48db-a0a2-2cf98e6d29bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:49.461879 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1ad92-d612-48db-a0a2-2cf98e6d29bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:49.470812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1ad92-d612-48db-a0a2-2cf98e6d29bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:49.472386 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1ad92-d612-48db-a0a2-2cf98e6d29bd service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:49.483417 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1251/2478] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:52:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:52:50.163838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c404b-e8e9-4895-b44a-b60737f09edf None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:50.165417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-805c404b-e8e9-4895-b44a-b60737f09edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:50.178509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c404b-e8e9-4895-b44a-b60737f09edf None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:50.184211 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e3338-5022-45b2-ad21-adc3b6ef50fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:50.185222 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e3338-5022-45b2-ad21-adc3b6ef50fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:50.185691 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e3338-5022-45b2-ad21-adc3b6ef50fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:50.185954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e3338-5022-45b2-ad21-adc3b6ef50fe admin admin] PATH_INFO: `/v3/projects/d881f1edc0404ed6827f5c8b62141837` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:50.187197 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e3338-5022-45b2-ad21-adc3b6ef50fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=d881f1edc0404ed6827f5c8b62141837)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:50.189180 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e3338-5022-45b2-ad21-adc3b6ef50fe admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:50.304075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-963e3338-5022-45b2-ad21-adc3b6ef50fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd881f1edc0404ed6827f5c8b62141837', 'request_id': 'req-963e3338-5022-45b2-ad21-adc3b6ef50fe'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:50.384949 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-963e3338-5022-45b2-ad21-adc3b6ef50fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd881f1edc0404ed6827f5c8b62141837', 'request_id': 'req-963e3338-5022-45b2-ad21-adc3b6ef50fe'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:50.394378 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-963e3338-5022-45b2-ad21-adc3b6ef50fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd881f1edc0404ed6827f5c8b62141837', 'request_id': 'req-963e3338-5022-45b2-ad21-adc3b6ef50fe'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:50.405882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-963e3338-5022-45b2-ad21-adc3b6ef50fe admin admin] The token cache is being invalidate because project d881f1edc0404ed6827f5c8b62141837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:52:50.406282 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-963e3338-5022-45b2-ad21-adc3b6ef50fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:50.407008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-963e3338-5022-45b2-ad21-adc3b6ef50fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:50.410306 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1228/2479] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:52:50 2022] DELETE /identity/v3/projects/d881f1edc0404ed6827f5c8b62141837 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:52:50.414223 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1ad92-d612-48db-a0a2-2cf98e6d29bd service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:50.425551 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1ad92-d612-48db-a0a2-2cf98e6d29bd service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:50.485822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffbdc4-0f3d-4873-a143-458a34b24372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:50.486870 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffbdc4-0f3d-4873-a143-458a34b24372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:50.487218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffbdc4-0f3d-4873-a143-458a34b24372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:50.487561 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffbdc4-0f3d-4873-a143-458a34b24372 admin admin] PATH_INFO: `/v3/users/24e2d97943aa47458db1f273a81c7b37` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:50.488930 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffbdc4-0f3d-4873-a143-458a34b24372 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24e2d97943aa47458db1f273a81c7b37)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:50.491002 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffbdc4-0f3d-4873-a143-458a34b24372 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:50.740621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cfffbdc4-0f3d-4873-a143-458a34b24372 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24e2d97943aa47458db1f273a81c7b37'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:50.753681 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cfffbdc4-0f3d-4873-a143-458a34b24372 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24e2d97943aa47458db1f273a81c7b37'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:50.754397 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cfffbdc4-0f3d-4873-a143-458a34b24372 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24e2d97943aa47458db1f273a81c7b37'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:50.755049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cfffbdc4-0f3d-4873-a143-458a34b24372 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24e2d97943aa47458db1f273a81c7b37'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:50.762451 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cfffbdc4-0f3d-4873-a143-458a34b24372 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24e2d97943aa47458db1f273a81c7b37'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:50.849017 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1252/2480] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:52:50 2022] DELETE /identity/v3/users/24e2d97943aa47458db1f273a81c7b37 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:52:51.423518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e3338-5022-45b2-ad21-adc3b6ef50fe admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:51.425860 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-963e3338-5022-45b2-ad21-adc3b6ef50fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:51.441440 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e3338-5022-45b2-ad21-adc3b6ef50fe admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:51.468189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:52:51.529439 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400e550-5cc4-4024-b5f4-a08c2192230f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:51.530252 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400e550-5cc4-4024-b5f4-a08c2192230f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:51.530533 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400e550-5cc4-4024-b5f4-a08c2192230f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:51.531023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400e550-5cc4-4024-b5f4-a08c2192230f admin admin] PATH_INFO: `/v3/projects/78f8d7e5f62741e48543f0cf89b6371d` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:51.532259 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400e550-5cc4-4024-b5f4-a08c2192230f admin admin] RBAC: Authorizing `identity:delete_project(project_id=78f8d7e5f62741e48543f0cf89b6371d)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:51.534539 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400e550-5cc4-4024-b5f4-a08c2192230f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:51.654864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a400e550-5cc4-4024-b5f4-a08c2192230f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78f8d7e5f62741e48543f0cf89b6371d', 'request_id': 'req-a400e550-5cc4-4024-b5f4-a08c2192230f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:51.712785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a400e550-5cc4-4024-b5f4-a08c2192230f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78f8d7e5f62741e48543f0cf89b6371d', 'request_id': 'req-a400e550-5cc4-4024-b5f4-a08c2192230f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:51.723732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a400e550-5cc4-4024-b5f4-a08c2192230f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78f8d7e5f62741e48543f0cf89b6371d', 'request_id': 'req-a400e550-5cc4-4024-b5f4-a08c2192230f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:51.737375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a400e550-5cc4-4024-b5f4-a08c2192230f admin admin] The token cache is being invalidate because project 78f8d7e5f62741e48543f0cf89b6371d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:52:51.738083 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a400e550-5cc4-4024-b5f4-a08c2192230f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:51.739598 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a400e550-5cc4-4024-b5f4-a08c2192230f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:51.741223 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1229/2481] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:52:51 2022] DELETE /identity/v3/projects/78f8d7e5f62741e48543f0cf89b6371d => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:52:51.746179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379d340-b1eb-4bde-aade-49319e88f2b0 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:51.747944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379d340-b1eb-4bde-aade-49319e88f2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:51.747944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379d340-b1eb-4bde-aade-49319e88f2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:51.764606 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-d379d340-b1eb-4bde-aade-49319e88f2b0 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:52:51.765828 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d379d340-b1eb-4bde-aade-49319e88f2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:51.844971 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1253/2482] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:52:51.845483 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa59bb-bfb5-4ef0-b8b6-7e7798229611 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:51.846357 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa59bb-bfb5-4ef0-b8b6-7e7798229611 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:51.846712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa59bb-bfb5-4ef0-b8b6-7e7798229611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:51.871931 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-b2fa59bb-bfb5-4ef0-b8b6-7e7798229611 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:52:51.874003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-b2fa59bb-bfb5-4ef0-b8b6-7e7798229611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:51.886314 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1230/2483] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:52:51.891368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379d340-b1eb-4bde-aade-49319e88f2b0 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:51.900959 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:52:51.905198 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379d340-b1eb-4bde-aade-49319e88f2b0 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:51.912076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e318d-f0ad-4418-8bec-2085ba930aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:51.914260 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e318d-f0ad-4418-8bec-2085ba930aba admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:51.914260 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e318d-f0ad-4418-8bec-2085ba930aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:51.914260 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e318d-f0ad-4418-8bec-2085ba930aba admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:51.914260 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e318d-f0ad-4418-8bec-2085ba930aba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:52:51.914669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e318d-f0ad-4418-8bec-2085ba930aba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:51.916164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e318d-f0ad-4418-8bec-2085ba930aba admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:51.928849 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1254/2484] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:52:51.933457 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa59bb-bfb5-4ef0-b8b6-7e7798229611 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:51.943419 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa59bb-bfb5-4ef0-b8b6-7e7798229611 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:51.948358 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e154fd5-04d6-4bfa-bee5-1fbe11da38f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:51.951639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e154fd5-04d6-4bfa-bee5-1fbe11da38f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:51.951639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e154fd5-04d6-4bfa-bee5-1fbe11da38f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:51.951639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e154fd5-04d6-4bfa-bee5-1fbe11da38f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:51.951639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e154fd5-04d6-4bfa-bee5-1fbe11da38f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:51.952418 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e154fd5-04d6-4bfa-bee5-1fbe11da38f5 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:52.063497 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1231/2485] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:52:51 2022] POST /identity/v3/projects => generated 359 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:52:52.068157 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e318d-f0ad-4418-8bec-2085ba930aba admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:52.080924 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e318d-f0ad-4418-8bec-2085ba930aba admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:52.159341 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d73e6-4ced-497d-b309-f612bcb4a99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:52.161016 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d73e6-4ced-497d-b309-f612bcb4a99f admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:52.161314 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d73e6-4ced-497d-b309-f612bcb4a99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:52.161599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d73e6-4ced-497d-b309-f612bcb4a99f admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:52.162349 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d73e6-4ced-497d-b309-f612bcb4a99f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:52.164401 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d73e6-4ced-497d-b309-f612bcb4a99f admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:52.279099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1255/2486] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:52:52 2022] POST /identity/v3/users => generated 280 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:52:52.284090 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e154fd5-04d6-4bfa-bee5-1fbe11da38f5 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:52.308633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e154fd5-04d6-4bfa-bee5-1fbe11da38f5 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:52.317476 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26ff98-4110-41aa-8d77-0d67c40d5623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:52.318086 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26ff98-4110-41aa-8d77-0d67c40d5623 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:52.318375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26ff98-4110-41aa-8d77-0d67c40d5623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:52.318661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26ff98-4110-41aa-8d77-0d67c40d5623 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:52.319402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26ff98-4110-41aa-8d77-0d67c40d5623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:52.321262 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26ff98-4110-41aa-8d77-0d67c40d5623 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:52.335810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1232/2487] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:52:52.339934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d73e6-4ced-497d-b309-f612bcb4a99f admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:52.350909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d73e6-4ced-497d-b309-f612bcb4a99f admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:52.361198 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66bb5c-9e8c-49b3-acc7-3c2f83be9078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:52.361198 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66bb5c-9e8c-49b3-acc7-3c2f83be9078 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:52.361198 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66bb5c-9e8c-49b3-acc7-3c2f83be9078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:52.361198 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66bb5c-9e8c-49b3-acc7-3c2f83be9078 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:52.361838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66bb5c-9e8c-49b3-acc7-3c2f83be9078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:52.361838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66bb5c-9e8c-49b3-acc7-3c2f83be9078 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:52.380288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1256/2488] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:52:52.381873 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26ff98-4110-41aa-8d77-0d67c40d5623 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:52.393706 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26ff98-4110-41aa-8d77-0d67c40d5623 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:52.398673 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20905fcb-8d72-430a-9513-3635b2cfbe9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:52.399568 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20905fcb-8d72-430a-9513-3635b2cfbe9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:52.399881 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20905fcb-8d72-430a-9513-3635b2cfbe9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:52.400162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20905fcb-8d72-430a-9513-3635b2cfbe9f admin admin] PATH_INFO: `/v3/projects/e8e86ad113ee402f805727dad113e633/users/8b28355d9a2d425fba8989d69e2347a0/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:52.416133 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20905fcb-8d72-430a-9513-3635b2cfbe9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8e86ad113ee402f805727dad113e633, user_id=8b28355d9a2d425fba8989d69e2347a0, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:52.419135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20905fcb-8d72-430a-9513-3635b2cfbe9f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:52.489077 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1233/2489] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:52:52 2022] PUT /identity/v3/projects/e8e86ad113ee402f805727dad113e633/users/8b28355d9a2d425fba8989d69e2347a0/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:52:52.499431 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9755d9-c72e-4201-9a86-2304ac34aab3 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:52.499737 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9755d9-c72e-4201-9a86-2304ac34aab3 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:52.500697 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9755d9-c72e-4201-9a86-2304ac34aab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:52.538428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-bb9755d9-c72e-4201-9a86-2304ac34aab3 None None] MFA Rules not processed for user `8b28355d9a2d425fba8989d69e2347a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:52:52.540076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-bb9755d9-c72e-4201-9a86-2304ac34aab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:52.610250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1257/2490] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:52:52 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:52:52.644835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20905fcb-8d72-430a-9513-3635b2cfbe9f admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:52.646824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-20905fcb-8d72-430a-9513-3635b2cfbe9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:52.652556 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9755d9-c72e-4201-9a86-2304ac34aab3 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:52.653981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-bb9755d9-c72e-4201-9a86-2304ac34aab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:52.703879 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:52:52.709317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20905fcb-8d72-430a-9513-3635b2cfbe9f admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:52.714775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc7721-17c1-4e32-9b51-681ba7f8a034 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:52.715436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc7721-17c1-4e32-9b51-681ba7f8a034 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:52.715723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc7721-17c1-4e32-9b51-681ba7f8a034 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:52.716002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc7721-17c1-4e32-9b51-681ba7f8a034 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:52.720662 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9755d9-c72e-4201-9a86-2304ac34aab3 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:52.724474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa1f01-684c-4952-a340-3e9a1e76a1e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:52.725202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa1f01-684c-4952-a340-3e9a1e76a1e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:52.725665 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa1f01-684c-4952-a340-3e9a1e76a1e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:52.725890 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dc7721-17c1-4e32-9b51-681ba7f8a034 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:52.726034 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa1f01-684c-4952-a340-3e9a1e76a1e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:52.727371 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dc7721-17c1-4e32-9b51-681ba7f8a034 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:52.728226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d5aa1f01-684c-4952-a340-3e9a1e76a1e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:52.741012 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1234/2491] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:52:52 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:52:52.749971 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:52:52.760185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc7721-17c1-4e32-9b51-681ba7f8a034 service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:52.802263 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa1f01-684c-4952-a340-3e9a1e76a1e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:52.803718 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa1f01-684c-4952-a340-3e9a1e76a1e2 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:52.813823 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1258/2492] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:52:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:52:52.824279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc7721-17c1-4e32-9b51-681ba7f8a034 service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:52.829196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76afae7-6c27-4e04-8071-e74a8893f900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:52.830101 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76afae7-6c27-4e04-8071-e74a8893f900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:52.830394 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76afae7-6c27-4e04-8071-e74a8893f900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:52.830675 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76afae7-6c27-4e04-8071-e74a8893f900 admin admin] PATH_INFO: `/v3/users/8b28355d9a2d425fba8989d69e2347a0` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:52.832211 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76afae7-6c27-4e04-8071-e74a8893f900 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b28355d9a2d425fba8989d69e2347a0)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:52.834213 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76afae7-6c27-4e04-8071-e74a8893f900 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:53.035938 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d76afae7-6c27-4e04-8071-e74a8893f900 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b28355d9a2d425fba8989d69e2347a0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:53.049439 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d76afae7-6c27-4e04-8071-e74a8893f900 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b28355d9a2d425fba8989d69e2347a0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:53.135764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d76afae7-6c27-4e04-8071-e74a8893f900 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b28355d9a2d425fba8989d69e2347a0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:53.135764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d76afae7-6c27-4e04-8071-e74a8893f900 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b28355d9a2d425fba8989d69e2347a0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:53.135764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d76afae7-6c27-4e04-8071-e74a8893f900 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b28355d9a2d425fba8989d69e2347a0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:53.149100 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1235/2493] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:52:52 2022] DELETE /identity/v3/users/8b28355d9a2d425fba8989d69e2347a0 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:52:53.155687 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665015c-9f6b-4e61-8ca1-81ab701e5ad7 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:53.156263 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665015c-9f6b-4e61-8ca1-81ab701e5ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:53.156800 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665015c-9f6b-4e61-8ca1-81ab701e5ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:53.181320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-a665015c-9f6b-4e61-8ca1-81ab701e5ad7 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:52:53.183049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a665015c-9f6b-4e61-8ca1-81ab701e5ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:53.260173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1259/2494] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:52:53.277711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76afae7-6c27-4e04-8071-e74a8893f900 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:53.279187 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-d76afae7-6c27-4e04-8071-e74a8893f900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:53.347462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76afae7-6c27-4e04-8071-e74a8893f900 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:53.351819 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d7e0c-10e5-4cce-a853-afe7489e101d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:53.352470 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d7e0c-10e5-4cce-a853-afe7489e101d service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:53.352783 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d7e0c-10e5-4cce-a853-afe7489e101d service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:53.353286 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d7e0c-10e5-4cce-a853-afe7489e101d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:53.364600 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d7e0c-10e5-4cce-a853-afe7489e101d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:53.365947 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d7e0c-10e5-4cce-a853-afe7489e101d service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:53.378613 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1236/2495] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:52:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:52:54.090819 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a665015c-9f6b-4e61-8ca1-81ab701e5ad7 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:54.092413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a665015c-9f6b-4e61-8ca1-81ab701e5ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:54.107723 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a665015c-9f6b-4e61-8ca1-81ab701e5ad7 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:54.114756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274687d3-6bfd-4d9c-b69f-4d8b314d9b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:54.116085 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274687d3-6bfd-4d9c-b69f-4d8b314d9b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:54.116697 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274687d3-6bfd-4d9c-b69f-4d8b314d9b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:54.116982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274687d3-6bfd-4d9c-b69f-4d8b314d9b8c admin admin] PATH_INFO: `/v3/projects/e8e86ad113ee402f805727dad113e633` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:54.118226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274687d3-6bfd-4d9c-b69f-4d8b314d9b8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8e86ad113ee402f805727dad113e633)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:54.120455 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274687d3-6bfd-4d9c-b69f-4d8b314d9b8c admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:54.327420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-274687d3-6bfd-4d9c-b69f-4d8b314d9b8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8e86ad113ee402f805727dad113e633', 'request_id': 'req-274687d3-6bfd-4d9c-b69f-4d8b314d9b8c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:54.335985 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-274687d3-6bfd-4d9c-b69f-4d8b314d9b8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8e86ad113ee402f805727dad113e633', 'request_id': 'req-274687d3-6bfd-4d9c-b69f-4d8b314d9b8c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:54.341791 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-274687d3-6bfd-4d9c-b69f-4d8b314d9b8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8e86ad113ee402f805727dad113e633', 'request_id': 'req-274687d3-6bfd-4d9c-b69f-4d8b314d9b8c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:54.427856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-274687d3-6bfd-4d9c-b69f-4d8b314d9b8c admin admin] The token cache is being invalidate because project e8e86ad113ee402f805727dad113e633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:52:54.428117 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-274687d3-6bfd-4d9c-b69f-4d8b314d9b8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:54.428828 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-274687d3-6bfd-4d9c-b69f-4d8b314d9b8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:52:54.430851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1260/2496] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:52:54 2022] DELETE /identity/v3/projects/e8e86ad113ee402f805727dad113e633 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:52:54.435748 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698dc90-09eb-4300-becf-d2ee6804acba None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:54.436199 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698dc90-09eb-4300-becf-d2ee6804acba None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:54.436303 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698dc90-09eb-4300-becf-d2ee6804acba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:54.455646 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-d698dc90-09eb-4300-becf-d2ee6804acba None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:52:54.456952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-d698dc90-09eb-4300-becf-d2ee6804acba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:54.466519 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:52:54.520993 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1237/2497] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:52:54.529856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654a688-c67c-4150-995c-4c439f402037 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:54.530150 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654a688-c67c-4150-995c-4c439f402037 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:54.530455 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654a688-c67c-4150-995c-4c439f402037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:54.548979 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-6654a688-c67c-4150-995c-4c439f402037 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:52:54.550270 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6654a688-c67c-4150-995c-4c439f402037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:54.558488 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1261/2498] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:52:54.562571 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698dc90-09eb-4300-becf-d2ee6804acba None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:54.573524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698dc90-09eb-4300-becf-d2ee6804acba None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:54.578360 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2675b8-2615-4353-b184-7c5718ed1b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:54.579125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2675b8-2615-4353-b184-7c5718ed1b22 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:54.579439 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2675b8-2615-4353-b184-7c5718ed1b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:54.579715 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2675b8-2615-4353-b184-7c5718ed1b22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:54.581344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2675b8-2615-4353-b184-7c5718ed1b22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:52:54.581519 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2675b8-2615-4353-b184-7c5718ed1b22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:54.583165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2675b8-2615-4353-b184-7c5718ed1b22 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:54.595809 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1238/2499] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:52:54.599871 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654a688-c67c-4150-995c-4c439f402037 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:54.609642 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654a688-c67c-4150-995c-4c439f402037 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:54.614977 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068323c-de5e-487c-a075-0cc4f210be24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:54.615644 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068323c-de5e-487c-a075-0cc4f210be24 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:54.615930 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068323c-de5e-487c-a075-0cc4f210be24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:54.616209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068323c-de5e-487c-a075-0cc4f210be24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:54.616990 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6068323c-de5e-487c-a075-0cc4f210be24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:54.618970 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6068323c-de5e-487c-a075-0cc4f210be24 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:54.707717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1262/2500] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:52:54 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:52:54.712399 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2675b8-2615-4353-b184-7c5718ed1b22 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:54.724421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2675b8-2615-4353-b184-7c5718ed1b22 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:54.782890 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b5dd8-1982-44e0-8efa-dc29bac224bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:54.783799 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b5dd8-1982-44e0-8efa-dc29bac224bb admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:54.784077 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b5dd8-1982-44e0-8efa-dc29bac224bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:54.784356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b5dd8-1982-44e0-8efa-dc29bac224bb admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:54.785097 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b5dd8-1982-44e0-8efa-dc29bac224bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:54.787101 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b5dd8-1982-44e0-8efa-dc29bac224bb admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:54.849970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1239/2501] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:52:54 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:52:54.854174 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068323c-de5e-487c-a075-0cc4f210be24 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:54.861810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:52:54.866634 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068323c-de5e-487c-a075-0cc4f210be24 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:54.871451 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2cb91a-8ae9-47d0-a7f4-ffbc2614f80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:54.873981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cb91a-8ae9-47d0-a7f4-ffbc2614f80a admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:54.873981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cb91a-8ae9-47d0-a7f4-ffbc2614f80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:54.873981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cb91a-8ae9-47d0-a7f4-ffbc2614f80a admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:54.873981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2cb91a-8ae9-47d0-a7f4-ffbc2614f80a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:54.875244 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2cb91a-8ae9-47d0-a7f4-ffbc2614f80a admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:54.887626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1263/2502] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:52:54.891677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b5dd8-1982-44e0-8efa-dc29bac224bb admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:54.903496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b5dd8-1982-44e0-8efa-dc29bac224bb admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:54.909211 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb3d89-8c31-4740-97c1-e3c378b8de3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:54.909915 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb3d89-8c31-4740-97c1-e3c378b8de3e admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:54.910209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb3d89-8c31-4740-97c1-e3c378b8de3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:54.910487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb3d89-8c31-4740-97c1-e3c378b8de3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:54.911213 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb3d89-8c31-4740-97c1-e3c378b8de3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:54.912873 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb3d89-8c31-4740-97c1-e3c378b8de3e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:54.923127 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1240/2503] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:52:54.927642 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2cb91a-8ae9-47d0-a7f4-ffbc2614f80a admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:54.938148 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2cb91a-8ae9-47d0-a7f4-ffbc2614f80a admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:54.943078 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850ad1a-04ca-4a47-9ed5-2d316f1775b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:54.944046 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850ad1a-04ca-4a47-9ed5-2d316f1775b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:54.944336 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850ad1a-04ca-4a47-9ed5-2d316f1775b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:54.944675 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850ad1a-04ca-4a47-9ed5-2d316f1775b7 admin admin] PATH_INFO: `/v3/projects/46af1aa5fcd24bac8e1b648ed4aee22a/users/2567c4b961e6433aa9919552a5cedab4/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:54.963365 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850ad1a-04ca-4a47-9ed5-2d316f1775b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46af1aa5fcd24bac8e1b648ed4aee22a, user_id=2567c4b961e6433aa9919552a5cedab4, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:54.966321 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850ad1a-04ca-4a47-9ed5-2d316f1775b7 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:55.067562 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1264/2504] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:52:54 2022] PUT /identity/v3/projects/46af1aa5fcd24bac8e1b648ed4aee22a/users/2567c4b961e6433aa9919552a5cedab4/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:52:55.072114 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a2982-e8f5-4435-a049-e009b99be8ee None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:55.078684 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a2982-e8f5-4435-a049-e009b99be8ee None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:55.078855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a2982-e8f5-4435-a049-e009b99be8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:55.111143 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-a24a2982-e8f5-4435-a049-e009b99be8ee None None] MFA Rules not processed for user `2567c4b961e6433aa9919552a5cedab4`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:52:55.113414 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-a24a2982-e8f5-4435-a049-e009b99be8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:55.194862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1241/2505] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:52:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:52:55.206288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850ad1a-04ca-4a47-9ed5-2d316f1775b7 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:55.208730 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-f850ad1a-04ca-4a47-9ed5-2d316f1775b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:55.303361 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850ad1a-04ca-4a47-9ed5-2d316f1775b7 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:55.310958 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8ae75-f184-44ad-9c71-26fcf78d8303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:55.312514 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8ae75-f184-44ad-9c71-26fcf78d8303 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:55.312963 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8ae75-f184-44ad-9c71-26fcf78d8303 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:55.313894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8ae75-f184-44ad-9c71-26fcf78d8303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:55.331496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8ae75-f184-44ad-9c71-26fcf78d8303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:55.331496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8ae75-f184-44ad-9c71-26fcf78d8303 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:55.338804 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1265/2506] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:52:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:52:56.436901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a2982-e8f5-4435-a049-e009b99be8ee None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:52:56.438289 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-a24a2982-e8f5-4435-a049-e009b99be8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:56.449861 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:52:56.496357 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a2982-e8f5-4435-a049-e009b99be8ee None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:52:56.500019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297411a8-5653-4ddc-b3dd-305748b43ca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:52:56.500703 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297411a8-5653-4ddc-b3dd-305748b43ca9 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:52:56.500992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297411a8-5653-4ddc-b3dd-305748b43ca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:52:56.501272 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297411a8-5653-4ddc-b3dd-305748b43ca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:52:56.503954 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-297411a8-5653-4ddc-b3dd-305748b43ca9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:52:56.568250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297411a8-5653-4ddc-b3dd-305748b43ca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:52:56.569727 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297411a8-5653-4ddc-b3dd-305748b43ca9 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:52:56.579470 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1242/2507] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:52:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:53:00.053380 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8ae75-f184-44ad-9c71-26fcf78d8303 service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:00.058230 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e7e8ae75-f184-44ad-9c71-26fcf78d8303 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:00.126104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8ae75-f184-44ad-9c71-26fcf78d8303 service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:00.131116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698bc44-cf9e-4d81-8400-f89250bc88f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ac44b3f0224e56a15993a2f29c1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:00.131758 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698bc44-cf9e-4d81-8400-f89250bc88f3 service placement] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:00.132041 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698bc44-cf9e-4d81-8400-f89250bc88f3 service placement] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:00.132316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698bc44-cf9e-4d81-8400-f89250bc88f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:00.134087 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-b698bc44-cf9e-4d81-8400-f89250bc88f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:00.146266 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b698bc44-cf9e-4d81-8400-f89250bc88f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:00.147852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b698bc44-cf9e-4d81-8400-f89250bc88f3 service placement] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:00.158637 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1266/2508] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:53:00.166423 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297411a8-5653-4ddc-b3dd-305748b43ca9 service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:00.168555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-297411a8-5653-4ddc-b3dd-305748b43ca9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:00.183862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297411a8-5653-4ddc-b3dd-305748b43ca9 service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:00.194261 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69658b04-ce2e-48a1-80be-99e521080907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:00.194261 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69658b04-ce2e-48a1-80be-99e521080907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:00.194261 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69658b04-ce2e-48a1-80be-99e521080907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:00.194261 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69658b04-ce2e-48a1-80be-99e521080907 admin admin] PATH_INFO: `/v3/users/5107e87ee4c34ef9ac96fa4a3a899bda` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:00.194639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69658b04-ce2e-48a1-80be-99e521080907 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5107e87ee4c34ef9ac96fa4a3a899bda)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:00.194639 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69658b04-ce2e-48a1-80be-99e521080907 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:00.398652 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-69658b04-ce2e-48a1-80be-99e521080907 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5107e87ee4c34ef9ac96fa4a3a899bda'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:00.410988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-69658b04-ce2e-48a1-80be-99e521080907 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5107e87ee4c34ef9ac96fa4a3a899bda'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:00.487318 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-69658b04-ce2e-48a1-80be-99e521080907 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5107e87ee4c34ef9ac96fa4a3a899bda'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:00.487930 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-69658b04-ce2e-48a1-80be-99e521080907 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5107e87ee4c34ef9ac96fa4a3a899bda'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:00.488568 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-69658b04-ce2e-48a1-80be-99e521080907 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5107e87ee4c34ef9ac96fa4a3a899bda'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:00.501123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1243/2509] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:53:00 2022] DELETE /identity/v3/users/5107e87ee4c34ef9ac96fa4a3a899bda => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:00.737454 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698bc44-cf9e-4d81-8400-f89250bc88f3 service placement] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:00.748997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698bc44-cf9e-4d81-8400-f89250bc88f3 service placement] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:00.816499 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a742e-b4fd-428d-a370-b0c18dc6f052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:00.817963 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a742e-b4fd-428d-a370-b0c18dc6f052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:00.817963 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a742e-b4fd-428d-a370-b0c18dc6f052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:00.818207 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a742e-b4fd-428d-a370-b0c18dc6f052 admin admin] PATH_INFO: `/v3/projects/13b00f92bcc449daadba8838f4c87a62` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:00.819619 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a742e-b4fd-428d-a370-b0c18dc6f052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13b00f92bcc449daadba8838f4c87a62)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:00.821588 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a742e-b4fd-428d-a370-b0c18dc6f052 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:00.824748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:00.954550 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d55a742e-b4fd-428d-a370-b0c18dc6f052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13b00f92bcc449daadba8838f4c87a62', 'request_id': 'req-d55a742e-b4fd-428d-a370-b0c18dc6f052'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:00.965093 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d55a742e-b4fd-428d-a370-b0c18dc6f052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13b00f92bcc449daadba8838f4c87a62', 'request_id': 'req-d55a742e-b4fd-428d-a370-b0c18dc6f052'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:00.970963 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d55a742e-b4fd-428d-a370-b0c18dc6f052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13b00f92bcc449daadba8838f4c87a62', 'request_id': 'req-d55a742e-b4fd-428d-a370-b0c18dc6f052'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:01.050745 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d55a742e-b4fd-428d-a370-b0c18dc6f052 admin admin] The token cache is being invalidate because project 13b00f92bcc449daadba8838f4c87a62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:53:01.051515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d55a742e-b4fd-428d-a370-b0c18dc6f052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:01.052335 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d55a742e-b4fd-428d-a370-b0c18dc6f052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:01.054444 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1267/2510] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:53:00 2022] DELETE /identity/v3/projects/13b00f92bcc449daadba8838f4c87a62 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:01.059647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6038b5-0584-4b32-a7f9-e58c1264f3ef None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:01.060014 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6038b5-0584-4b32-a7f9-e58c1264f3ef None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:01.060367 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6038b5-0584-4b32-a7f9-e58c1264f3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:01.080199 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-2a6038b5-0584-4b32-a7f9-e58c1264f3ef None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:01.081986 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-2a6038b5-0584-4b32-a7f9-e58c1264f3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:01.109870 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:01.151025 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1244/2511] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:01.152805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a56350-4541-4ec5-8697-8c0b419c503c None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:01.154923 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a56350-4541-4ec5-8697-8c0b419c503c None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:01.154923 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a56350-4541-4ec5-8697-8c0b419c503c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:01.172695 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-f2a56350-4541-4ec5-8697-8c0b419c503c None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:01.173937 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-f2a56350-4541-4ec5-8697-8c0b419c503c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:01.182399 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1268/2512] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:53:01.186741 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6038b5-0584-4b32-a7f9-e58c1264f3ef None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:01.196263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6038b5-0584-4b32-a7f9-e58c1264f3ef None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:01.200934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365190f-73d0-4b8f-8f0d-e8b23dcde2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:01.201594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365190f-73d0-4b8f-8f0d-e8b23dcde2a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:01.201880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365190f-73d0-4b8f-8f0d-e8b23dcde2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:01.202161 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365190f-73d0-4b8f-8f0d-e8b23dcde2a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:01.204148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365190f-73d0-4b8f-8f0d-e8b23dcde2a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:53:01.204463 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365190f-73d0-4b8f-8f0d-e8b23dcde2a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:01.206616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365190f-73d0-4b8f-8f0d-e8b23dcde2a3 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:01.218592 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1245/2513] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:53:01.226155 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a56350-4541-4ec5-8697-8c0b419c503c None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:01.235816 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a56350-4541-4ec5-8697-8c0b419c503c None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:01.240648 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0092ee1-6138-42b3-bb35-d274288d8d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:01.241797 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0092ee1-6138-42b3-bb35-d274288d8d0b admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:01.242360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0092ee1-6138-42b3-bb35-d274288d8d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:01.243179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0092ee1-6138-42b3-bb35-d274288d8d0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:01.243907 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0092ee1-6138-42b3-bb35-d274288d8d0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:01.245869 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0092ee1-6138-42b3-bb35-d274288d8d0b admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:01.360990 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1269/2514] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:53:01 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:01.365109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365190f-73d0-4b8f-8f0d-e8b23dcde2a3 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:01.384360 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365190f-73d0-4b8f-8f0d-e8b23dcde2a3 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:01.488778 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc4262-8980-468f-ab87-924e687106f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:01.490612 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc4262-8980-468f-ab87-924e687106f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:01.490612 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc4262-8980-468f-ab87-924e687106f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:01.492960 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc4262-8980-468f-ab87-924e687106f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:01.494352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc4262-8980-468f-ab87-924e687106f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:01.496500 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc4262-8980-468f-ab87-924e687106f1 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:01.588186 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1246/2515] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:53:01 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:01.588693 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0092ee1-6138-42b3-bb35-d274288d8d0b admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:01.603338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0092ee1-6138-42b3-bb35-d274288d8d0b admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:01.609020 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d01b5-55da-4ec2-ae27-1f99169ad2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:01.609705 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d01b5-55da-4ec2-ae27-1f99169ad2bf admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:01.609987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d01b5-55da-4ec2-ae27-1f99169ad2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:01.610266 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d01b5-55da-4ec2-ae27-1f99169ad2bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:01.611118 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d01b5-55da-4ec2-ae27-1f99169ad2bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:01.613400 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d01b5-55da-4ec2-ae27-1f99169ad2bf admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:01.624800 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1270/2516] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:01.629697 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc4262-8980-468f-ab87-924e687106f1 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:01.641678 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc4262-8980-468f-ab87-924e687106f1 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:01.649719 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af056461-1a59-4f98-a953-3d13a9975dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:01.649858 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af056461-1a59-4f98-a953-3d13a9975dda admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:01.650114 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af056461-1a59-4f98-a953-3d13a9975dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:01.650397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af056461-1a59-4f98-a953-3d13a9975dda admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:01.651583 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af056461-1a59-4f98-a953-3d13a9975dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:01.653354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af056461-1a59-4f98-a953-3d13a9975dda admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:01.668074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1247/2517] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:01.673070 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d01b5-55da-4ec2-ae27-1f99169ad2bf admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:01.688644 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d01b5-55da-4ec2-ae27-1f99169ad2bf admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:01.698760 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c913783-95d6-45ad-99d9-024c97b92ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:01.702728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c913783-95d6-45ad-99d9-024c97b92ccc admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:01.702728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c913783-95d6-45ad-99d9-024c97b92ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:01.702728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c913783-95d6-45ad-99d9-024c97b92ccc admin admin] PATH_INFO: `/v3/projects/c438364dd80f44cd90ae5c2b9228a094/users/58fd3740e2ba4bfc90abce2b44cc2f64/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:01.731805 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c913783-95d6-45ad-99d9-024c97b92ccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c438364dd80f44cd90ae5c2b9228a094, user_id=58fd3740e2ba4bfc90abce2b44cc2f64, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:01.734649 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c913783-95d6-45ad-99d9-024c97b92ccc admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:01.830186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1271/2518] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:53:01 2022] PUT /identity/v3/projects/c438364dd80f44cd90ae5c2b9228a094/users/58fd3740e2ba4bfc90abce2b44cc2f64/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:01.840903 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ace05-e2cf-48d3-aba0-7fa50e0ec933 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:01.841200 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ace05-e2cf-48d3-aba0-7fa50e0ec933 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:01.841510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ace05-e2cf-48d3-aba0-7fa50e0ec933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:01.879108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-ac7ace05-e2cf-48d3-aba0-7fa50e0ec933 None None] MFA Rules not processed for user `58fd3740e2ba4bfc90abce2b44cc2f64`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:01.880409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ac7ace05-e2cf-48d3-aba0-7fa50e0ec933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:01.985013 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1248/2519] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:01.997374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c913783-95d6-45ad-99d9-024c97b92ccc admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:01.999171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6c913783-95d6-45ad-99d9-024c97b92ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:02.130727 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c913783-95d6-45ad-99d9-024c97b92ccc admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:02.137180 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb3366-daca-40e7-9ada-969e1a6513e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:02.137824 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb3366-daca-40e7-9ada-969e1a6513e2 service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:02.138164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb3366-daca-40e7-9ada-969e1a6513e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:02.138433 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb3366-daca-40e7-9ada-969e1a6513e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:02.148383 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb3366-daca-40e7-9ada-969e1a6513e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:02.150702 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb3366-daca-40e7-9ada-969e1a6513e2 service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:02.164776 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1272/2520] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:53:11.981152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=85,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:12.159563 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:13.600672 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ace05-e2cf-48d3-aba0-7fa50e0ec933 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:13.660814 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ace05-e2cf-48d3-aba0-7fa50e0ec933 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:13.665519 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f570d2f-3865-4bb8-91d5-53a0f1e04ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:13.666389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f570d2f-3865-4bb8-91d5-53a0f1e04ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:13.666672 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f570d2f-3865-4bb8-91d5-53a0f1e04ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:13.666948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f570d2f-3865-4bb8-91d5-53a0f1e04ef3 admin admin] PATH_INFO: `/v3/users/58fd3740e2ba4bfc90abce2b44cc2f64` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:13.668125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f570d2f-3865-4bb8-91d5-53a0f1e04ef3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58fd3740e2ba4bfc90abce2b44cc2f64)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:13.670113 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f570d2f-3865-4bb8-91d5-53a0f1e04ef3 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:13.814833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3f570d2f-3865-4bb8-91d5-53a0f1e04ef3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58fd3740e2ba4bfc90abce2b44cc2f64'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:13.827342 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3f570d2f-3865-4bb8-91d5-53a0f1e04ef3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58fd3740e2ba4bfc90abce2b44cc2f64'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:13.900737 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3f570d2f-3865-4bb8-91d5-53a0f1e04ef3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58fd3740e2ba4bfc90abce2b44cc2f64'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:13.901361 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3f570d2f-3865-4bb8-91d5-53a0f1e04ef3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58fd3740e2ba4bfc90abce2b44cc2f64'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:13.902754 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3f570d2f-3865-4bb8-91d5-53a0f1e04ef3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58fd3740e2ba4bfc90abce2b44cc2f64'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:13.915920 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1249/2521] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:53:13 2022] DELETE /identity/v3/users/58fd3740e2ba4bfc90abce2b44cc2f64 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:13.920667 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b5620-3f90-42ba-894c-245e521273c7 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:13.925754 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b5620-3f90-42ba-894c-245e521273c7 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:13.925754 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b5620-3f90-42ba-894c-245e521273c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:13.939151 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-6f3b5620-3f90-42ba-894c-245e521273c7 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:13.940365 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6f3b5620-3f90-42ba-894c-245e521273c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:14.003468 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1273/2522] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:14.013242 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f570d2f-3865-4bb8-91d5-53a0f1e04ef3 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:14.014636 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3f570d2f-3865-4bb8-91d5-53a0f1e04ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:14.071206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f570d2f-3865-4bb8-91d5-53a0f1e04ef3 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:14.074837 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00843ad7-5569-4ca5-b6ad-6d9b2b706d36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:14.075481 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00843ad7-5569-4ca5-b6ad-6d9b2b706d36 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:14.075767 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00843ad7-5569-4ca5-b6ad-6d9b2b706d36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:14.076051 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00843ad7-5569-4ca5-b6ad-6d9b2b706d36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:14.085121 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00843ad7-5569-4ca5-b6ad-6d9b2b706d36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:14.086616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00843ad7-5569-4ca5-b6ad-6d9b2b706d36 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:14.096048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1250/2523] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:53:14.488405 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b5620-3f90-42ba-894c-245e521273c7 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:14.491333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6f3b5620-3f90-42ba-894c-245e521273c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:14.502433 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b5620-3f90-42ba-894c-245e521273c7 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:14.507273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c912a-1137-49a7-b6b3-f1604195af56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:14.508138 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c912a-1137-49a7-b6b3-f1604195af56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:14.508428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c912a-1137-49a7-b6b3-f1604195af56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:14.508739 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c912a-1137-49a7-b6b3-f1604195af56 admin admin] PATH_INFO: `/v3/projects/c438364dd80f44cd90ae5c2b9228a094` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:14.509926 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c912a-1137-49a7-b6b3-f1604195af56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c438364dd80f44cd90ae5c2b9228a094)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:14.511888 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c912a-1137-49a7-b6b3-f1604195af56 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:14.603522 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b01c912a-1137-49a7-b6b3-f1604195af56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c438364dd80f44cd90ae5c2b9228a094', 'request_id': 'req-b01c912a-1137-49a7-b6b3-f1604195af56'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:14.609925 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b01c912a-1137-49a7-b6b3-f1604195af56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c438364dd80f44cd90ae5c2b9228a094', 'request_id': 'req-b01c912a-1137-49a7-b6b3-f1604195af56'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:14.615240 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b01c912a-1137-49a7-b6b3-f1604195af56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c438364dd80f44cd90ae5c2b9228a094', 'request_id': 'req-b01c912a-1137-49a7-b6b3-f1604195af56'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:14.680257 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b01c912a-1137-49a7-b6b3-f1604195af56 admin admin] The token cache is being invalidate because project c438364dd80f44cd90ae5c2b9228a094 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:53:14.680547 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b01c912a-1137-49a7-b6b3-f1604195af56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:14.681184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b01c912a-1137-49a7-b6b3-f1604195af56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:14.682716 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1274/2524] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:53:14 2022] DELETE /identity/v3/projects/c438364dd80f44cd90ae5c2b9228a094 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:14.687730 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c19f6-e956-4276-800f-2077f99b7813 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:14.688227 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c19f6-e956-4276-800f-2077f99b7813 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:14.691984 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c19f6-e956-4276-800f-2077f99b7813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:14.707714 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-2f8c19f6-e956-4276-800f-2077f99b7813 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:14.709651 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-2f8c19f6-e956-4276-800f-2077f99b7813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:14.726555 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:14.767711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1251/2525] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:14.776230 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed28fd7-b42c-42b7-9d53-03d208bb5598 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:14.776526 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed28fd7-b42c-42b7-9d53-03d208bb5598 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:14.776835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed28fd7-b42c-42b7-9d53-03d208bb5598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:14.794993 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-4ed28fd7-b42c-42b7-9d53-03d208bb5598 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:14.796197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-4ed28fd7-b42c-42b7-9d53-03d208bb5598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:14.804147 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1275/2526] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:53:14.808377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c19f6-e956-4276-800f-2077f99b7813 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:14.817920 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c19f6-e956-4276-800f-2077f99b7813 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:14.823492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696580e-2422-4a27-bb29-7f3e7b7d613f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:14.824135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696580e-2422-4a27-bb29-7f3e7b7d613f admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:14.824419 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696580e-2422-4a27-bb29-7f3e7b7d613f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:14.824727 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696580e-2422-4a27-bb29-7f3e7b7d613f admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:14.825451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696580e-2422-4a27-bb29-7f3e7b7d613f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:53:14.825755 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696580e-2422-4a27-bb29-7f3e7b7d613f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:14.827535 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696580e-2422-4a27-bb29-7f3e7b7d613f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:14.840360 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1252/2527] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:53:14.844372 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed28fd7-b42c-42b7-9d53-03d208bb5598 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:14.853756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed28fd7-b42c-42b7-9d53-03d208bb5598 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:14.858621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc29d2-8efe-41f2-ab5b-2b3660ac6eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:14.859262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc29d2-8efe-41f2-ab5b-2b3660ac6eb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:14.859553 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc29d2-8efe-41f2-ab5b-2b3660ac6eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:14.859832 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc29d2-8efe-41f2-ab5b-2b3660ac6eb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:14.860622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc29d2-8efe-41f2-ab5b-2b3660ac6eb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:14.862493 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc29d2-8efe-41f2-ab5b-2b3660ac6eb2 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:14.939716 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1276/2528] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:53:14 2022] POST /identity/v3/projects => generated 361 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:14.944255 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696580e-2422-4a27-bb29-7f3e7b7d613f admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:14.954569 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696580e-2422-4a27-bb29-7f3e7b7d613f admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:15.016489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2eb6b9-cf8e-476b-9e01-63883f40662f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:15.017230 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2eb6b9-cf8e-476b-9e01-63883f40662f admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:15.017545 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2eb6b9-cf8e-476b-9e01-63883f40662f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:15.017963 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2eb6b9-cf8e-476b-9e01-63883f40662f admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:15.018654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2eb6b9-cf8e-476b-9e01-63883f40662f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:15.020563 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2eb6b9-cf8e-476b-9e01-63883f40662f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:15.134123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1253/2529] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:53:14 2022] POST /identity/v3/users => generated 281 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:15.138217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc29d2-8efe-41f2-ab5b-2b3660ac6eb2 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:15.148320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc29d2-8efe-41f2-ab5b-2b3660ac6eb2 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:15.152948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675aca84-4a04-464d-992f-b3820ae44e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:15.153604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675aca84-4a04-464d-992f-b3820ae44e07 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:15.153883 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675aca84-4a04-464d-992f-b3820ae44e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:15.154156 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675aca84-4a04-464d-992f-b3820ae44e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:15.154877 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675aca84-4a04-464d-992f-b3820ae44e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:15.156516 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675aca84-4a04-464d-992f-b3820ae44e07 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:15.166811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1277/2530] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:15.170916 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2eb6b9-cf8e-476b-9e01-63883f40662f admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:15.181646 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2eb6b9-cf8e-476b-9e01-63883f40662f admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:15.186201 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1afb726-0d31-4615-9611-a324fb923d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:15.186869 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afb726-0d31-4615-9611-a324fb923d2c admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:15.187152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afb726-0d31-4615-9611-a324fb923d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:15.187428 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afb726-0d31-4615-9611-a324fb923d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:15.188173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1afb726-0d31-4615-9611-a324fb923d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:15.189812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1afb726-0d31-4615-9611-a324fb923d2c admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:15.202673 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1254/2531] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:15.206755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675aca84-4a04-464d-992f-b3820ae44e07 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:15.216439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675aca84-4a04-464d-992f-b3820ae44e07 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:15.220947 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e4c0f-c817-4e96-8a99-8307c5b0d6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:15.221809 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e4c0f-c817-4e96-8a99-8307c5b0d6a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:15.222092 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e4c0f-c817-4e96-8a99-8307c5b0d6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:15.222366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e4c0f-c817-4e96-8a99-8307c5b0d6a2 admin admin] PATH_INFO: `/v3/projects/ac5db55c768341bbbe6d090b8c81eeca/users/521df9b7e3924be9adb3a7b0ec5dff7d/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:15.236922 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e4c0f-c817-4e96-8a99-8307c5b0d6a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac5db55c768341bbbe6d090b8c81eeca, user_id=521df9b7e3924be9adb3a7b0ec5dff7d, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:15.239555 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e4c0f-c817-4e96-8a99-8307c5b0d6a2 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:15.318170 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1278/2532] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:53:15 2022] PUT /identity/v3/projects/ac5db55c768341bbbe6d090b8c81eeca/users/521df9b7e3924be9adb3a7b0ec5dff7d/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:15.330117 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3544d8-0c0f-4559-a0ba-c69b46df543c None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:15.330416 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3544d8-0c0f-4559-a0ba-c69b46df543c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:15.330704 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3544d8-0c0f-4559-a0ba-c69b46df543c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:15.357211 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-3b3544d8-0c0f-4559-a0ba-c69b46df543c None None] MFA Rules not processed for user `521df9b7e3924be9adb3a7b0ec5dff7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:15.358425 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3b3544d8-0c0f-4559-a0ba-c69b46df543c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:15.423105 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1255/2533] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:15.428085 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e4c0f-c817-4e96-8a99-8307c5b0d6a2 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:15.449528 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:15.489375 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e4c0f-c817-4e96-8a99-8307c5b0d6a2 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:15.494700 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25eb272-28fc-4aec-b41c-dd445885a636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:15.495353 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25eb272-28fc-4aec-b41c-dd445885a636 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:15.495638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25eb272-28fc-4aec-b41c-dd445885a636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:15.495916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25eb272-28fc-4aec-b41c-dd445885a636 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:15.496671 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25eb272-28fc-4aec-b41c-dd445885a636 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:15.498925 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25eb272-28fc-4aec-b41c-dd445885a636 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:15.559600 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1279/2534] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:53:15 2022] POST /identity/v3/projects => generated 365 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:15.563415 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3544d8-0c0f-4559-a0ba-c69b46df543c None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:15.574460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3544d8-0c0f-4559-a0ba-c69b46df543c None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:15.586933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:15.632411 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44f74e-0c40-4f36-a333-185965afa24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:15.633145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44f74e-0c40-4f36-a333-185965afa24e admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:15.633439 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44f74e-0c40-4f36-a333-185965afa24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:15.633721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44f74e-0c40-4f36-a333-185965afa24e admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:15.634413 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44f74e-0c40-4f36-a333-185965afa24e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:15.636367 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44f74e-0c40-4f36-a333-185965afa24e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:15.751790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1256/2535] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:53:15 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:15.755955 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25eb272-28fc-4aec-b41c-dd445885a636 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:15.766406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25eb272-28fc-4aec-b41c-dd445885a636 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:15.770997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f926ab4-214a-400e-be3d-ae11e6c8f79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:15.772463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f926ab4-214a-400e-be3d-ae11e6c8f79b admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:15.772775 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f926ab4-214a-400e-be3d-ae11e6c8f79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:15.773063 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f926ab4-214a-400e-be3d-ae11e6c8f79b admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:15.773822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f926ab4-214a-400e-be3d-ae11e6c8f79b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:15.775424 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f926ab4-214a-400e-be3d-ae11e6c8f79b admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:15.785844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1280/2536] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:15.790589 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44f74e-0c40-4f36-a333-185965afa24e admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:15.800458 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44f74e-0c40-4f36-a333-185965afa24e admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:15.805481 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15206c-564e-41aa-8f66-72abeda23fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:15.806356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15206c-564e-41aa-8f66-72abeda23fac admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:15.806641 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15206c-564e-41aa-8f66-72abeda23fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:15.806922 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15206c-564e-41aa-8f66-72abeda23fac admin admin] PATH_INFO: `/v3/projects/f42e317b638546e5ad9315f523a9213f/users/b61b59e1569c4cc5b342c586bc9e412d/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:15.821470 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f15206c-564e-41aa-8f66-72abeda23fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f42e317b638546e5ad9315f523a9213f, user_id=b61b59e1569c4cc5b342c586bc9e412d, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:15.824136 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f15206c-564e-41aa-8f66-72abeda23fac admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:15.918395 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1257/2537] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:53:15 2022] PUT /identity/v3/projects/f42e317b638546e5ad9315f523a9213f/users/b61b59e1569c4cc5b342c586bc9e412d/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:15.922526 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f926ab4-214a-400e-be3d-ae11e6c8f79b admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:15.932748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f926ab4-214a-400e-be3d-ae11e6c8f79b admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:15.988240 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38a14b-98df-453e-a058-4023c0c4608a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:15.988911 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38a14b-98df-453e-a058-4023c0c4608a admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:15.989194 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38a14b-98df-453e-a058-4023c0c4608a admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:15.989472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38a14b-98df-453e-a058-4023c0c4608a admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:15.990238 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38a14b-98df-453e-a058-4023c0c4608a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:15.991830 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38a14b-98df-453e-a058-4023c0c4608a admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:16.001575 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1281/2538] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:16.005574 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15206c-564e-41aa-8f66-72abeda23fac admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:16.015759 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15206c-564e-41aa-8f66-72abeda23fac admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:16.020969 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d67f59-edec-43ef-ada8-8f00be64f884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:16.022621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d67f59-edec-43ef-ada8-8f00be64f884 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:16.022904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d67f59-edec-43ef-ada8-8f00be64f884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:16.023184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d67f59-edec-43ef-ada8-8f00be64f884 admin admin] PATH_INFO: `/v3/projects/f42e317b638546e5ad9315f523a9213f/users/b61b59e1569c4cc5b342c586bc9e412d/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:16.025214 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d67f59-edec-43ef-ada8-8f00be64f884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f42e317b638546e5ad9315f523a9213f, user_id=b61b59e1569c4cc5b342c586bc9e412d, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:16.027857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d67f59-edec-43ef-ada8-8f00be64f884 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:16.039087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1258/2539] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:53:16 2022] PUT /identity/v3/projects/f42e317b638546e5ad9315f523a9213f/users/b61b59e1569c4cc5b342c586bc9e412d/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:16.048996 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f8957-f5ac-417c-9785-8feb2cc54817 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:16.049293 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f8957-f5ac-417c-9785-8feb2cc54817 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:16.049577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f8957-f5ac-417c-9785-8feb2cc54817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:16.075594 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-e80f8957-f5ac-417c-9785-8feb2cc54817 None None] MFA Rules not processed for user `b61b59e1569c4cc5b342c586bc9e412d`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:16.076837 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e80f8957-f5ac-417c-9785-8feb2cc54817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:16.145698 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1282/2540] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:16 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:16.157786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d67f59-edec-43ef-ada8-8f00be64f884 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:16.159571 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-27d67f59-edec-43ef-ada8-8f00be64f884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:16.219217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d67f59-edec-43ef-ada8-8f00be64f884 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:16.223032 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aba5bb-2d00-490b-a7c0-af6a60f19894 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:16.223748 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aba5bb-2d00-490b-a7c0-af6a60f19894 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:16.224035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aba5bb-2d00-490b-a7c0-af6a60f19894 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:16.224313 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aba5bb-2d00-490b-a7c0-af6a60f19894 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:16.237699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aba5bb-2d00-490b-a7c0-af6a60f19894 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:16.239164 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aba5bb-2d00-490b-a7c0-af6a60f19894 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:16.250128 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1259/2541] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:16 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:53:17.173353 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f8957-f5ac-417c-9785-8feb2cc54817 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:17.191187 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:17.238349 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f8957-f5ac-417c-9785-8feb2cc54817 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:17.243005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85ff32-9c1a-4361-96ec-9a13e12c70f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:17.243954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85ff32-9c1a-4361-96ec-9a13e12c70f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:17.244246 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85ff32-9c1a-4361-96ec-9a13e12c70f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:17.244551 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85ff32-9c1a-4361-96ec-9a13e12c70f6 admin admin] PATH_INFO: `/v3/users/521df9b7e3924be9adb3a7b0ec5dff7d` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:17.245782 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85ff32-9c1a-4361-96ec-9a13e12c70f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=521df9b7e3924be9adb3a7b0ec5dff7d)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:17.247709 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85ff32-9c1a-4361-96ec-9a13e12c70f6 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:17.495280 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-1e85ff32-9c1a-4361-96ec-9a13e12c70f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '521df9b7e3924be9adb3a7b0ec5dff7d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:17.509079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-1e85ff32-9c1a-4361-96ec-9a13e12c70f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '521df9b7e3924be9adb3a7b0ec5dff7d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:17.515018 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-1e85ff32-9c1a-4361-96ec-9a13e12c70f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '521df9b7e3924be9adb3a7b0ec5dff7d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:17.515018 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-1e85ff32-9c1a-4361-96ec-9a13e12c70f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '521df9b7e3924be9adb3a7b0ec5dff7d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:17.517435 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-1e85ff32-9c1a-4361-96ec-9a13e12c70f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '521df9b7e3924be9adb3a7b0ec5dff7d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:17.626491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1283/2542] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:53:17 2022] DELETE /identity/v3/users/521df9b7e3924be9adb3a7b0ec5dff7d => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:17.631811 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93a485-553a-4565-a1f3-a41cad3c5926 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:17.632318 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93a485-553a-4565-a1f3-a41cad3c5926 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:17.632436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93a485-553a-4565-a1f3-a41cad3c5926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:17.649964 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-1b93a485-553a-4565-a1f3-a41cad3c5926 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:17.651161 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-1b93a485-553a-4565-a1f3-a41cad3c5926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:17.713727 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1260/2543] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:17.733389 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85ff32-9c1a-4361-96ec-9a13e12c70f6 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:17.735366 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-1e85ff32-9c1a-4361-96ec-9a13e12c70f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:17.799521 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85ff32-9c1a-4361-96ec-9a13e12c70f6 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:17.806065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d922b-46ba-450a-8971-d2ebf22ecec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:17.806775 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d922b-46ba-450a-8971-d2ebf22ecec8 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:17.807067 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d922b-46ba-450a-8971-d2ebf22ecec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:17.807344 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d922b-46ba-450a-8971-d2ebf22ecec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:17.817479 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d922b-46ba-450a-8971-d2ebf22ecec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:17.819037 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d922b-46ba-450a-8971-d2ebf22ecec8 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:17.829333 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1284/2544] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:53:18.239320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93a485-553a-4565-a1f3-a41cad3c5926 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:18.242354 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-1b93a485-553a-4565-a1f3-a41cad3c5926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:18.254796 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93a485-553a-4565-a1f3-a41cad3c5926 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:18.260070 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3bb03-ebbb-4349-a731-b286c1451723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:18.261148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3bb03-ebbb-4349-a731-b286c1451723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:18.261432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3bb03-ebbb-4349-a731-b286c1451723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:18.261707 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3bb03-ebbb-4349-a731-b286c1451723 admin admin] PATH_INFO: `/v3/projects/ac5db55c768341bbbe6d090b8c81eeca` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:18.262897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3bb03-ebbb-4349-a731-b286c1451723 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac5db55c768341bbbe6d090b8c81eeca)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:18.265116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3bb03-ebbb-4349-a731-b286c1451723 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:18.274252 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:18.387571 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-aba3bb03-ebbb-4349-a731-b286c1451723 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac5db55c768341bbbe6d090b8c81eeca', 'request_id': 'req-aba3bb03-ebbb-4349-a731-b286c1451723'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:18.449491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-aba3bb03-ebbb-4349-a731-b286c1451723 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac5db55c768341bbbe6d090b8c81eeca', 'request_id': 'req-aba3bb03-ebbb-4349-a731-b286c1451723'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:18.456540 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-aba3bb03-ebbb-4349-a731-b286c1451723 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac5db55c768341bbbe6d090b8c81eeca', 'request_id': 'req-aba3bb03-ebbb-4349-a731-b286c1451723'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:18.467930 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-aba3bb03-ebbb-4349-a731-b286c1451723 admin admin] The token cache is being invalidate because project ac5db55c768341bbbe6d090b8c81eeca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:53:18.468219 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-aba3bb03-ebbb-4349-a731-b286c1451723 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:18.468894 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-aba3bb03-ebbb-4349-a731-b286c1451723 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:18.470526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1261/2545] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:53:18 2022] DELETE /identity/v3/projects/ac5db55c768341bbbe6d090b8c81eeca => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:18.475169 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d922b-46ba-450a-8971-d2ebf22ecec8 service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:18.485023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d922b-46ba-450a-8971-d2ebf22ecec8 service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:18.518341 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:18.544209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa81e2-45ba-4cb4-b2ba-6db901df57ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:18.545208 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa81e2-45ba-4cb4-b2ba-6db901df57ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:18.545494 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa81e2-45ba-4cb4-b2ba-6db901df57ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:18.545861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa81e2-45ba-4cb4-b2ba-6db901df57ad admin admin] PATH_INFO: `/v3/users/b61b59e1569c4cc5b342c586bc9e412d` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:18.547251 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa81e2-45ba-4cb4-b2ba-6db901df57ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=b61b59e1569c4cc5b342c586bc9e412d)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:18.549257 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa81e2-45ba-4cb4-b2ba-6db901df57ad admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:18.553333 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe6da5-1787-4783-9d27-3363142e4763 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:18.555692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe6da5-1787-4783-9d27-3363142e4763 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:18.555692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe6da5-1787-4783-9d27-3363142e4763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:18.572217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-d9fe6da5-1787-4783-9d27-3363142e4763 None None] MFA Rules not processed for user `0a7c204a298c42f0bbc32fa9a6ce1f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:18.573495 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-d9fe6da5-1787-4783-9d27-3363142e4763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:18.627591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1262/2546] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 23 22:53:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:53:18.759824 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e3fa81e2-45ba-4cb4-b2ba-6db901df57ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b61b59e1569c4cc5b342c586bc9e412d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:18.762061 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe6da5-1787-4783-9d27-3363142e4763 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:18.763511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-d9fe6da5-1787-4783-9d27-3363142e4763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:18.773531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e3fa81e2-45ba-4cb4-b2ba-6db901df57ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b61b59e1569c4cc5b342c586bc9e412d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:18.774232 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e3fa81e2-45ba-4cb4-b2ba-6db901df57ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b61b59e1569c4cc5b342c586bc9e412d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:18.774847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e3fa81e2-45ba-4cb4-b2ba-6db901df57ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b61b59e1569c4cc5b342c586bc9e412d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:18.783061 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e3fa81e2-45ba-4cb4-b2ba-6db901df57ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b61b59e1569c4cc5b342c586bc9e412d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:18.836300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe6da5-1787-4783-9d27-3363142e4763 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:18.840365 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2d26d-3c61-4586-9f52-1e9eea520753 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b321586cdb3d4d25904472aaa4adba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:18.841059 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2d26d-3c61-4586-9f52-1e9eea520753 service swift] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:18.841329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2d26d-3c61-4586-9f52-1e9eea520753 service swift] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:18.841677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2d26d-3c61-4586-9f52-1e9eea520753 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:18.843485 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-f5f2d26d-3c61-4586-9f52-1e9eea520753 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:18.855428 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2d26d-3c61-4586-9f52-1e9eea520753 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:18.857052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2d26d-3c61-4586-9f52-1e9eea520753 service swift] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:18.863118 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1285/2547] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:53:18 2022] DELETE /identity/v3/users/b61b59e1569c4cc5b342c586bc9e412d => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:18.866427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1263/2548] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 22:53:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 22:53:19.302400 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa81e2-45ba-4cb4-b2ba-6db901df57ad admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:19.304037 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e3fa81e2-45ba-4cb4-b2ba-6db901df57ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:19.376794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa81e2-45ba-4cb4-b2ba-6db901df57ad admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:19.382236 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd1201a-b50d-4297-9075-672f84314bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:19.383127 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1201a-b50d-4297-9075-672f84314bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:19.383436 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1201a-b50d-4297-9075-672f84314bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:19.383706 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1201a-b50d-4297-9075-672f84314bff admin admin] PATH_INFO: `/v3/projects/f42e317b638546e5ad9315f523a9213f` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:19.384905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd1201a-b50d-4297-9075-672f84314bff admin admin] RBAC: Authorizing `identity:delete_project(project_id=f42e317b638546e5ad9315f523a9213f)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:19.386791 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd1201a-b50d-4297-9075-672f84314bff admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:19.492143 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3cd1201a-b50d-4297-9075-672f84314bff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f42e317b638546e5ad9315f523a9213f', 'request_id': 'req-3cd1201a-b50d-4297-9075-672f84314bff'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:19.498666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3cd1201a-b50d-4297-9075-672f84314bff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f42e317b638546e5ad9315f523a9213f', 'request_id': 'req-3cd1201a-b50d-4297-9075-672f84314bff'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:19.502918 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:19.505962 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3cd1201a-b50d-4297-9075-672f84314bff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f42e317b638546e5ad9315f523a9213f', 'request_id': 'req-3cd1201a-b50d-4297-9075-672f84314bff'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:19.577542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3cd1201a-b50d-4297-9075-672f84314bff admin admin] The token cache is being invalidate because project f42e317b638546e5ad9315f523a9213f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:53:19.577767 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3cd1201a-b50d-4297-9075-672f84314bff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:19.578511 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-3cd1201a-b50d-4297-9075-672f84314bff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:19.580186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1286/2549] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:53:19 2022] DELETE /identity/v3/projects/f42e317b638546e5ad9315f523a9213f => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:19.585229 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd4b67-f71f-44df-92b8-2ea9e9b04c78 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:19.585535 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd4b67-f71f-44df-92b8-2ea9e9b04c78 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:19.585817 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd4b67-f71f-44df-92b8-2ea9e9b04c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:19.604500 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-63bd4b67-f71f-44df-92b8-2ea9e9b04c78 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:19.605748 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-63bd4b67-f71f-44df-92b8-2ea9e9b04c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:19.634394 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:19.668699 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1264/2550] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:19.676649 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c4c03-ce41-4439-86ea-31cc4dc5a4fa None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:19.676945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c4c03-ce41-4439-86ea-31cc4dc5a4fa None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:19.677856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c4c03-ce41-4439-86ea-31cc4dc5a4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:19.698389 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-b74c4c03-ce41-4439-86ea-31cc4dc5a4fa None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:19.699851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-b74c4c03-ce41-4439-86ea-31cc4dc5a4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:19.709966 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1287/2551] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:19.714473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd4b67-f71f-44df-92b8-2ea9e9b04c78 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:19.724165 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd4b67-f71f-44df-92b8-2ea9e9b04c78 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:19.729504 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5919036-282c-41da-a78d-5f346d05c03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:19.730174 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5919036-282c-41da-a78d-5f346d05c03e admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:19.730478 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5919036-282c-41da-a78d-5f346d05c03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:19.730860 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5919036-282c-41da-a78d-5f346d05c03e admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:19.731594 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5919036-282c-41da-a78d-5f346d05c03e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:53:19.731908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5919036-282c-41da-a78d-5f346d05c03e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:19.733561 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5919036-282c-41da-a78d-5f346d05c03e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:19.746359 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1265/2552] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:53:19.750858 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c4c03-ce41-4439-86ea-31cc4dc5a4fa None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:19.760729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c4c03-ce41-4439-86ea-31cc4dc5a4fa None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:19.765323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7828c7-b59a-480b-8f4a-f6838ff32d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:19.770107 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7828c7-b59a-480b-8f4a-f6838ff32d37 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:19.770107 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7828c7-b59a-480b-8f4a-f6838ff32d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:19.770107 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7828c7-b59a-480b-8f4a-f6838ff32d37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:19.770107 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7828c7-b59a-480b-8f4a-f6838ff32d37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:19.770107 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7828c7-b59a-480b-8f4a-f6838ff32d37 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:19.858546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1288/2553] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:53:19 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:19.863334 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5919036-282c-41da-a78d-5f346d05c03e admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:19.874597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5919036-282c-41da-a78d-5f346d05c03e admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:19.935896 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138e7a1-d494-4a5b-9a9e-5247ce7307f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:19.936565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138e7a1-d494-4a5b-9a9e-5247ce7307f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:19.936879 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138e7a1-d494-4a5b-9a9e-5247ce7307f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:19.937157 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138e7a1-d494-4a5b-9a9e-5247ce7307f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:19.937838 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138e7a1-d494-4a5b-9a9e-5247ce7307f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:19.939753 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138e7a1-d494-4a5b-9a9e-5247ce7307f3 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:20.068303 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1266/2554] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:53:19 2022] POST /identity/v3/users => generated 285 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:20.069860 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7828c7-b59a-480b-8f4a-f6838ff32d37 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:20.082202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7828c7-b59a-480b-8f4a-f6838ff32d37 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:20.087278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a860d3d-aa39-401f-9b9b-a6186e18a295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:20.088156 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a860d3d-aa39-401f-9b9b-a6186e18a295 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:20.088671 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a860d3d-aa39-401f-9b9b-a6186e18a295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:20.089111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a860d3d-aa39-401f-9b9b-a6186e18a295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:20.090032 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a860d3d-aa39-401f-9b9b-a6186e18a295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:20.091953 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a860d3d-aa39-401f-9b9b-a6186e18a295 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:20.103973 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1289/2555] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:53:20.109986 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138e7a1-d494-4a5b-9a9e-5247ce7307f3 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:20.120850 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138e7a1-d494-4a5b-9a9e-5247ce7307f3 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:20.125642 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d430e-a767-4794-ad4b-ecfed055191c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:20.126310 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d430e-a767-4794-ad4b-ecfed055191c admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:20.126597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d430e-a767-4794-ad4b-ecfed055191c admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:20.132525 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d430e-a767-4794-ad4b-ecfed055191c admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:20.132525 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d430e-a767-4794-ad4b-ecfed055191c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:20.132525 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d430e-a767-4794-ad4b-ecfed055191c admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:20.139806 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1267/2556] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:20.143998 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a860d3d-aa39-401f-9b9b-a6186e18a295 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:20.154458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a860d3d-aa39-401f-9b9b-a6186e18a295 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:20.159116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eed812-e3bd-4360-9ccc-d10cf7450d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:20.159996 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eed812-e3bd-4360-9ccc-d10cf7450d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:20.160279 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eed812-e3bd-4360-9ccc-d10cf7450d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:20.160610 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eed812-e3bd-4360-9ccc-d10cf7450d0a admin admin] PATH_INFO: `/v3/projects/385c21202aec4dacbb6eac46653bab66/users/d23bf8e26c2f46108d6969e5a43a23e3/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:20.176951 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eed812-e3bd-4360-9ccc-d10cf7450d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=385c21202aec4dacbb6eac46653bab66, user_id=d23bf8e26c2f46108d6969e5a43a23e3, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:20.180352 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eed812-e3bd-4360-9ccc-d10cf7450d0a admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:20.295852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1290/2557] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:53:20 2022] PUT /identity/v3/projects/385c21202aec4dacbb6eac46653bab66/users/d23bf8e26c2f46108d6969e5a43a23e3/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:20.306836 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1bcfe-f394-4b74-b873-f463b3caa38d None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:20.307132 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1bcfe-f394-4b74-b873-f463b3caa38d None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:20.307436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1bcfe-f394-4b74-b873-f463b3caa38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:20.336402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-0cc1bcfe-f394-4b74-b873-f463b3caa38d None None] MFA Rules not processed for user `d23bf8e26c2f46108d6969e5a43a23e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:20.338132 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-0cc1bcfe-f394-4b74-b873-f463b3caa38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:20.429932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1268/2558] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:20 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:20.434551 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eed812-e3bd-4360-9ccc-d10cf7450d0a admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:20.506110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eed812-e3bd-4360-9ccc-d10cf7450d0a admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:20.510988 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620866a4-a95d-40a2-8dcf-8f7f9f6277e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:20.511645 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620866a4-a95d-40a2-8dcf-8f7f9f6277e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:20.511933 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620866a4-a95d-40a2-8dcf-8f7f9f6277e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:20.512214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620866a4-a95d-40a2-8dcf-8f7f9f6277e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:20.512941 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620866a4-a95d-40a2-8dcf-8f7f9f6277e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:20.514881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620866a4-a95d-40a2-8dcf-8f7f9f6277e5 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:20.598561 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1291/2559] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:53:20 2022] POST /identity/v3/projects => generated 365 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:20.602816 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1bcfe-f394-4b74-b873-f463b3caa38d None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:20.617656 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1bcfe-f394-4b74-b873-f463b3caa38d None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:20.638820 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:20.672314 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cb679-4d93-4c6b-b587-9ca33707e6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:20.673063 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cb679-4d93-4c6b-b587-9ca33707e6f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:20.673460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cb679-4d93-4c6b-b587-9ca33707e6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:20.673749 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cb679-4d93-4c6b-b587-9ca33707e6f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:20.674512 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78cb679-4d93-4c6b-b587-9ca33707e6f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:20.677375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78cb679-4d93-4c6b-b587-9ca33707e6f8 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:20.756043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1269/2560] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:53:20 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:20.757354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620866a4-a95d-40a2-8dcf-8f7f9f6277e5 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:20.767708 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620866a4-a95d-40a2-8dcf-8f7f9f6277e5 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:20.773502 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09dc7e-3960-4961-8386-5f8834d4b635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:20.774016 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09dc7e-3960-4961-8386-5f8834d4b635 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:20.774304 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09dc7e-3960-4961-8386-5f8834d4b635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:20.774584 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09dc7e-3960-4961-8386-5f8834d4b635 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:20.775323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09dc7e-3960-4961-8386-5f8834d4b635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:20.781695 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09dc7e-3960-4961-8386-5f8834d4b635 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:20.796374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1292/2561] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:53:20.799445 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cb679-4d93-4c6b-b587-9ca33707e6f8 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:20.810022 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cb679-4d93-4c6b-b587-9ca33707e6f8 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:20.815248 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3571398c-4750-4579-8436-61c2bd7b01c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:20.816196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3571398c-4750-4579-8436-61c2bd7b01c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:20.816480 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3571398c-4750-4579-8436-61c2bd7b01c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:20.816934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3571398c-4750-4579-8436-61c2bd7b01c6 admin admin] PATH_INFO: `/v3/projects/668c9933534a45f5a216e34d34717952/users/ec6fda27a78045a8bfbb58ebbcede225/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:20.835915 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3571398c-4750-4579-8436-61c2bd7b01c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=668c9933534a45f5a216e34d34717952, user_id=ec6fda27a78045a8bfbb58ebbcede225, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:20.838790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3571398c-4750-4579-8436-61c2bd7b01c6 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:20.919513 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1270/2562] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:53:20 2022] PUT /identity/v3/projects/668c9933534a45f5a216e34d34717952/users/ec6fda27a78045a8bfbb58ebbcede225/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:53:20.923515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09dc7e-3960-4961-8386-5f8834d4b635 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:20.933903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09dc7e-3960-4961-8386-5f8834d4b635 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:20.985452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:20.991310 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35962d86-b290-440d-9714-c867e3a69156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:20.991981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35962d86-b290-440d-9714-c867e3a69156 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:20.992270 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35962d86-b290-440d-9714-c867e3a69156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:20.992553 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35962d86-b290-440d-9714-c867e3a69156 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:20.993307 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35962d86-b290-440d-9714-c867e3a69156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:20.994904 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35962d86-b290-440d-9714-c867e3a69156 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:21.007579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1293/2563] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:21.009851 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3571398c-4750-4579-8436-61c2bd7b01c6 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:21.023101 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3571398c-4750-4579-8436-61c2bd7b01c6 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:21.028247 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69805fa0-d9ea-4163-93a6-b4794a6ee1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:21.029274 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69805fa0-d9ea-4163-93a6-b4794a6ee1e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:21.029646 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69805fa0-d9ea-4163-93a6-b4794a6ee1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:21.030462 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69805fa0-d9ea-4163-93a6-b4794a6ee1e6 admin admin] PATH_INFO: `/v3/projects/668c9933534a45f5a216e34d34717952/users/ec6fda27a78045a8bfbb58ebbcede225/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:21.032382 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69805fa0-d9ea-4163-93a6-b4794a6ee1e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=668c9933534a45f5a216e34d34717952, user_id=ec6fda27a78045a8bfbb58ebbcede225, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:21.035397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69805fa0-d9ea-4163-93a6-b4794a6ee1e6 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:21.050316 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1271/2564] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:53:21 2022] PUT /identity/v3/projects/668c9933534a45f5a216e34d34717952/users/ec6fda27a78045a8bfbb58ebbcede225/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:21.061631 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4692f3-2e18-4e08-a7a0-70326e3b47dd None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:21.061931 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4692f3-2e18-4e08-a7a0-70326e3b47dd None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:21.062213 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4692f3-2e18-4e08-a7a0-70326e3b47dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:21.107861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-bd4692f3-2e18-4e08-a7a0-70326e3b47dd None None] MFA Rules not processed for user `ec6fda27a78045a8bfbb58ebbcede225`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:21.109368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-bd4692f3-2e18-4e08-a7a0-70326e3b47dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:21.200765 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1294/2565] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:21 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:53:21.209560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69805fa0-d9ea-4163-93a6-b4794a6ee1e6 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:21.212321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-69805fa0-d9ea-4163-93a6-b4794a6ee1e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:21.286138 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69805fa0-d9ea-4163-93a6-b4794a6ee1e6 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:21.290044 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583792dc-19a0-4617-ab76-8a217648260a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:21.290961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583792dc-19a0-4617-ab76-8a217648260a service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:21.291359 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583792dc-19a0-4617-ab76-8a217648260a service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:21.291734 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583792dc-19a0-4617-ab76-8a217648260a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:21.301844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583792dc-19a0-4617-ab76-8a217648260a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:21.303959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583792dc-19a0-4617-ab76-8a217648260a service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:21.314501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1272/2566] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:21 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:53:21.731439 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4692f3-2e18-4e08-a7a0-70326e3b47dd None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:21.733713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-bd4692f3-2e18-4e08-a7a0-70326e3b47dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:21.810025 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4692f3-2e18-4e08-a7a0-70326e3b47dd None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:21.815839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8c0b5-f69d-451a-b97f-7c09933ece00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:21.816762 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8c0b5-f69d-451a-b97f-7c09933ece00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:21.817055 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8c0b5-f69d-451a-b97f-7c09933ece00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:21.817337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8c0b5-f69d-451a-b97f-7c09933ece00 admin admin] PATH_INFO: `/v3/users/2567c4b961e6433aa9919552a5cedab4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:21.818755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8c0b5-f69d-451a-b97f-7c09933ece00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2567c4b961e6433aa9919552a5cedab4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:21.820757 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8c0b5-f69d-451a-b97f-7c09933ece00 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:22.028336 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-66f8c0b5-f69d-451a-b97f-7c09933ece00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2567c4b961e6433aa9919552a5cedab4'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:22.041719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-66f8c0b5-f69d-451a-b97f-7c09933ece00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2567c4b961e6433aa9919552a5cedab4'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:22.045054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-66f8c0b5-f69d-451a-b97f-7c09933ece00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2567c4b961e6433aa9919552a5cedab4'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:22.045054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-66f8c0b5-f69d-451a-b97f-7c09933ece00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2567c4b961e6433aa9919552a5cedab4'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:22.050224 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-66f8c0b5-f69d-451a-b97f-7c09933ece00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2567c4b961e6433aa9919552a5cedab4'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:22.240373 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:22.244461 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1295/2567] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:53:21 2022] DELETE /identity/v3/users/2567c4b961e6433aa9919552a5cedab4 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:53:22.250451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eccf4b-f5b0-41f4-8341-2dbf7c3eb4c2 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:22.252231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eccf4b-f5b0-41f4-8341-2dbf7c3eb4c2 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:22.252777 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eccf4b-f5b0-41f4-8341-2dbf7c3eb4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:22.291561 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-16eccf4b-f5b0-41f4-8341-2dbf7c3eb4c2 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:22.294422 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-16eccf4b-f5b0-41f4-8341-2dbf7c3eb4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:22.383303 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1273/2568] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:22.393596 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8c0b5-f69d-451a-b97f-7c09933ece00 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:22.395222 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-66f8c0b5-f69d-451a-b97f-7c09933ece00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:22.478287 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8c0b5-f69d-451a-b97f-7c09933ece00 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:22.482186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba524263-427b-42cb-ba32-4a605ffe633e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:22.485851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba524263-427b-42cb-ba32-4a605ffe633e service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:22.485851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba524263-427b-42cb-ba32-4a605ffe633e service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:22.485851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba524263-427b-42cb-ba32-4a605ffe633e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:22.493463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba524263-427b-42cb-ba32-4a605ffe633e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:22.494897 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba524263-427b-42cb-ba32-4a605ffe633e service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:22.504634 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1296/2569] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:53:22.659935 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eccf4b-f5b0-41f4-8341-2dbf7c3eb4c2 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:22.662054 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-16eccf4b-f5b0-41f4-8341-2dbf7c3eb4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:22.690333 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:22.757833 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eccf4b-f5b0-41f4-8341-2dbf7c3eb4c2 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:22.771485 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d6763-b986-46a1-af5d-4dd7489399ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ac44b3f0224e56a15993a2f29c1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:22.773525 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d6763-b986-46a1-af5d-4dd7489399ac service placement] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:22.777114 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d6763-b986-46a1-af5d-4dd7489399ac service placement] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:22.777114 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d6763-b986-46a1-af5d-4dd7489399ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:22.780155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c72d6763-b986-46a1-af5d-4dd7489399ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:22.804871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d6763-b986-46a1-af5d-4dd7489399ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:22.806742 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d6763-b986-46a1-af5d-4dd7489399ac service placement] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:22.834566 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1274/2570] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:53:23.014399 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba524263-427b-42cb-ba32-4a605ffe633e service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:23.021328 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ba524263-427b-42cb-ba32-4a605ffe633e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:23.033799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba524263-427b-42cb-ba32-4a605ffe633e service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:23.039982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b15c27-993c-4544-af37-b1fac60eea13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:23.041336 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b15c27-993c-4544-af37-b1fac60eea13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:23.041633 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b15c27-993c-4544-af37-b1fac60eea13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:23.041838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b15c27-993c-4544-af37-b1fac60eea13 admin admin] PATH_INFO: `/v3/projects/46af1aa5fcd24bac8e1b648ed4aee22a` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:23.043249 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b15c27-993c-4544-af37-b1fac60eea13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46af1aa5fcd24bac8e1b648ed4aee22a)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:23.045484 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b15c27-993c-4544-af37-b1fac60eea13 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:23.227130 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d1b15c27-993c-4544-af37-b1fac60eea13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46af1aa5fcd24bac8e1b648ed4aee22a', 'request_id': 'req-d1b15c27-993c-4544-af37-b1fac60eea13'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:23.237492 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d1b15c27-993c-4544-af37-b1fac60eea13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46af1aa5fcd24bac8e1b648ed4aee22a', 'request_id': 'req-d1b15c27-993c-4544-af37-b1fac60eea13'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:23.245259 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d1b15c27-993c-4544-af37-b1fac60eea13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46af1aa5fcd24bac8e1b648ed4aee22a', 'request_id': 'req-d1b15c27-993c-4544-af37-b1fac60eea13'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:23.344956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d1b15c27-993c-4544-af37-b1fac60eea13 admin admin] The token cache is being invalidate because project 46af1aa5fcd24bac8e1b648ed4aee22a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:53:23.347566 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d1b15c27-993c-4544-af37-b1fac60eea13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:23.347566 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d1b15c27-993c-4544-af37-b1fac60eea13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:23.348331 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1297/2571] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:53:23 2022] DELETE /identity/v3/projects/46af1aa5fcd24bac8e1b648ed4aee22a => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:23.354062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8624e-aa67-487c-923c-27c544a7cffc None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:23.354831 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8624e-aa67-487c-923c-27c544a7cffc None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:23.354949 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8624e-aa67-487c-923c-27c544a7cffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:23.379307 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-c1b8624e-aa67-487c-923c-27c544a7cffc None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:23.380242 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c1b8624e-aa67-487c-923c-27c544a7cffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:23.463099 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1275/2572] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:53:23.470669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e72d3e-ced9-4e31-af06-60cbe26edcc9 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:23.471050 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e72d3e-ced9-4e31-af06-60cbe26edcc9 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:23.471411 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e72d3e-ced9-4e31-af06-60cbe26edcc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:23.498242 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-72e72d3e-ced9-4e31-af06-60cbe26edcc9 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:23.500961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-72e72d3e-ced9-4e31-af06-60cbe26edcc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:23.514850 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1298/2573] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:23.519123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8624e-aa67-487c-923c-27c544a7cffc None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:23.530343 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8624e-aa67-487c-923c-27c544a7cffc None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:23.540389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66cdf07-0f06-4881-8e7f-ec5d5273c094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:23.540389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cdf07-0f06-4881-8e7f-ec5d5273c094 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:23.540389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cdf07-0f06-4881-8e7f-ec5d5273c094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:23.540389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cdf07-0f06-4881-8e7f-ec5d5273c094 admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:23.542619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66cdf07-0f06-4881-8e7f-ec5d5273c094 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:53:23.542619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66cdf07-0f06-4881-8e7f-ec5d5273c094 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:23.542619 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66cdf07-0f06-4881-8e7f-ec5d5273c094 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:23.576998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1276/2574] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:53:23.578938 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e72d3e-ced9-4e31-af06-60cbe26edcc9 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:23.604567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e72d3e-ced9-4e31-af06-60cbe26edcc9 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:23.611932 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8609ae-5a8a-4778-8f6d-f3c466264c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:23.612968 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8609ae-5a8a-4778-8f6d-f3c466264c2a admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:23.613275 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8609ae-5a8a-4778-8f6d-f3c466264c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:23.613530 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8609ae-5a8a-4778-8f6d-f3c466264c2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:23.614467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8609ae-5a8a-4778-8f6d-f3c466264c2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:23.618628 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8609ae-5a8a-4778-8f6d-f3c466264c2a admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:23.737012 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1299/2575] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:53:23 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:23.738860 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66cdf07-0f06-4881-8e7f-ec5d5273c094 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:23.756264 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66cdf07-0f06-4881-8e7f-ec5d5273c094 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:23.833936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f60fd6-2801-48ad-a914-2e85e205ce4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:23.834743 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f60fd6-2801-48ad-a914-2e85e205ce4b admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:23.835024 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f60fd6-2801-48ad-a914-2e85e205ce4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:23.835303 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f60fd6-2801-48ad-a914-2e85e205ce4b admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:23.835990 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f60fd6-2801-48ad-a914-2e85e205ce4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:23.838330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f60fd6-2801-48ad-a914-2e85e205ce4b admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:23.927336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1277/2576] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:53:23 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:23.931446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8609ae-5a8a-4778-8f6d-f3c466264c2a admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:23.954373 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8609ae-5a8a-4778-8f6d-f3c466264c2a admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:23.962835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a93a7-8208-4fe7-ae2e-77d245e446d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:23.963615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a93a7-8208-4fe7-ae2e-77d245e446d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:23.963980 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a93a7-8208-4fe7-ae2e-77d245e446d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:23.964340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a93a7-8208-4fe7-ae2e-77d245e446d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:23.965221 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a93a7-8208-4fe7-ae2e-77d245e446d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:23.967106 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a93a7-8208-4fe7-ae2e-77d245e446d2 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:23.981420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1300/2577] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:23.984189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f60fd6-2801-48ad-a914-2e85e205ce4b admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:23.996929 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f60fd6-2801-48ad-a914-2e85e205ce4b admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:24.010458 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d51e46-bddd-4efb-b3d7-a5465a4a325f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:24.010458 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d51e46-bddd-4efb-b3d7-a5465a4a325f admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:24.010458 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d51e46-bddd-4efb-b3d7-a5465a4a325f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:24.010458 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d51e46-bddd-4efb-b3d7-a5465a4a325f admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:24.010853 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d51e46-bddd-4efb-b3d7-a5465a4a325f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:24.010853 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d51e46-bddd-4efb-b3d7-a5465a4a325f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:24.022132 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1278/2578] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:24.026421 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a93a7-8208-4fe7-ae2e-77d245e446d2 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:24.036509 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a93a7-8208-4fe7-ae2e-77d245e446d2 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:24.041651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21f247-9f09-4150-9346-0c44e26d28b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:24.042534 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21f247-9f09-4150-9346-0c44e26d28b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:24.042903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21f247-9f09-4150-9346-0c44e26d28b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:24.043353 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21f247-9f09-4150-9346-0c44e26d28b6 admin admin] PATH_INFO: `/v3/projects/112025b81f6f4005ad293939972f0576/users/8744ee74dd4143b39a87a04405b8dc9b/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:24.059248 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21f247-9f09-4150-9346-0c44e26d28b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=112025b81f6f4005ad293939972f0576, user_id=8744ee74dd4143b39a87a04405b8dc9b, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:24.062093 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21f247-9f09-4150-9346-0c44e26d28b6 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:24.144345 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1301/2579] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:53:24 2022] PUT /identity/v3/projects/112025b81f6f4005ad293939972f0576/users/8744ee74dd4143b39a87a04405b8dc9b/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:24.154832 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024d8ae-af48-4baa-8cf1-8272b300242d None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:24.155180 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024d8ae-af48-4baa-8cf1-8272b300242d None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:24.155403 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024d8ae-af48-4baa-8cf1-8272b300242d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:24.188321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-d024d8ae-af48-4baa-8cf1-8272b300242d None None] MFA Rules not processed for user `8744ee74dd4143b39a87a04405b8dc9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:24.189841 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-d024d8ae-af48-4baa-8cf1-8272b300242d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:24.197476 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:24.262229 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1279/2580] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:24.266854 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21f247-9f09-4150-9346-0c44e26d28b6 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:24.297410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:24.344662 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21f247-9f09-4150-9346-0c44e26d28b6 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:24.350717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c116c-4c20-4f8f-9c89-459500117333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:24.351477 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c116c-4c20-4f8f-9c89-459500117333 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:24.351768 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c116c-4c20-4f8f-9c89-459500117333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:24.352044 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c116c-4c20-4f8f-9c89-459500117333 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:24.352782 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c116c-4c20-4f8f-9c89-459500117333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:24.355106 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c116c-4c20-4f8f-9c89-459500117333 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:24.492135 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1302/2581] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:53:24 2022] POST /identity/v3/projects => generated 385 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:24.496280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024d8ae-af48-4baa-8cf1-8272b300242d None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:24.509346 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024d8ae-af48-4baa-8cf1-8272b300242d None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:24.567960 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9e46d-50f8-405b-9f0a-978c65844a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:24.568991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9e46d-50f8-405b-9f0a-978c65844a4a admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:24.569298 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9e46d-50f8-405b-9f0a-978c65844a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:24.569580 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9e46d-50f8-405b-9f0a-978c65844a4a admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:24.570272 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9e46d-50f8-405b-9f0a-978c65844a4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:24.572196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9e46d-50f8-405b-9f0a-978c65844a4a admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:24.692774 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1280/2582] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:53:24 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:24.696956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c116c-4c20-4f8f-9c89-459500117333 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:24.707494 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c116c-4c20-4f8f-9c89-459500117333 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:24.712349 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fae987-8ed3-4dc3-8c14-805df87a9920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:24.713487 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fae987-8ed3-4dc3-8c14-805df87a9920 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:24.714197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fae987-8ed3-4dc3-8c14-805df87a9920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:24.720810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fae987-8ed3-4dc3-8c14-805df87a9920 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:24.720810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fae987-8ed3-4dc3-8c14-805df87a9920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:24.720810 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fae987-8ed3-4dc3-8c14-805df87a9920 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:24.748994 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9e46d-50f8-405b-9f0a-978c65844a4a admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:24.751106 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-6ef9e46d-50f8-405b-9f0a-978c65844a4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:24.762459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1303/2583] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:53:24.763456 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fae987-8ed3-4dc3-8c14-805df87a9920 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:24.768799 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9e46d-50f8-405b-9f0a-978c65844a4a admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:24.774940 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9864c-eddf-44e2-9b98-4a66392c6051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:24.776117 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9864c-eddf-44e2-9b98-4a66392c6051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:24.776648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9864c-eddf-44e2-9b98-4a66392c6051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:24.777160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9864c-eddf-44e2-9b98-4a66392c6051 admin admin] PATH_INFO: `/v3/users/d23bf8e26c2f46108d6969e5a43a23e3` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:24.778621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9864c-eddf-44e2-9b98-4a66392c6051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d23bf8e26c2f46108d6969e5a43a23e3)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:24.780190 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fae987-8ed3-4dc3-8c14-805df87a9920 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:24.781280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9864c-eddf-44e2-9b98-4a66392c6051 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:24.789601 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02da70c-3306-4d5a-916d-25915a9479fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:24.790642 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02da70c-3306-4d5a-916d-25915a9479fd admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:24.790937 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02da70c-3306-4d5a-916d-25915a9479fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:24.791224 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02da70c-3306-4d5a-916d-25915a9479fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:24.791971 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02da70c-3306-4d5a-916d-25915a9479fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:24.793691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02da70c-3306-4d5a-916d-25915a9479fd admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:24.807425 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1304/2584] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:24.812847 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02da70c-3306-4d5a-916d-25915a9479fd admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:24.828150 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02da70c-3306-4d5a-916d-25915a9479fd admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:24.836737 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cfaeb8-c9ca-4b89-bce7-3871d6e7720d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:24.837660 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfaeb8-c9ca-4b89-bce7-3871d6e7720d admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:24.837948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfaeb8-c9ca-4b89-bce7-3871d6e7720d admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:24.838285 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfaeb8-c9ca-4b89-bce7-3871d6e7720d admin admin] PATH_INFO: `/v3/projects/49d65cdfe62e44e796942a744b5117e1/users/5de2ea4a29fe4cd38ad5509a35f8016b/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:24.857926 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cfaeb8-c9ca-4b89-bce7-3871d6e7720d admin admin] RBAC: Authorizing `identity:create_grant(project_id=49d65cdfe62e44e796942a744b5117e1, user_id=5de2ea4a29fe4cd38ad5509a35f8016b, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:24.861293 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cfaeb8-c9ca-4b89-bce7-3871d6e7720d admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:24.978669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1305/2585] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:53:24 2022] PUT /identity/v3/projects/49d65cdfe62e44e796942a744b5117e1/users/5de2ea4a29fe4cd38ad5509a35f8016b/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:24.987815 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbe56c-8af3-463b-b01d-129d5c148251 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:24.988116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbe56c-8af3-463b-b01d-129d5c148251 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:24.988402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbe56c-8af3-463b-b01d-129d5c148251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:25.014858 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-0fe9864c-eddf-44e2-9b98-4a66392c6051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd23bf8e26c2f46108d6969e5a43a23e3'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:25.018378 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-46bbe56c-8af3-463b-b01d-129d5c148251 None None] MFA Rules not processed for user `5de2ea4a29fe4cd38ad5509a35f8016b`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:25.019694 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-46bbe56c-8af3-463b-b01d-129d5c148251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:25.030385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-0fe9864c-eddf-44e2-9b98-4a66392c6051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd23bf8e26c2f46108d6969e5a43a23e3'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:25.088845 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1306/2586] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:53:25.093156 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbe56c-8af3-463b-b01d-129d5c148251 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:25.109158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-0fe9864c-eddf-44e2-9b98-4a66392c6051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd23bf8e26c2f46108d6969e5a43a23e3'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:25.109908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-0fe9864c-eddf-44e2-9b98-4a66392c6051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd23bf8e26c2f46108d6969e5a43a23e3'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:25.110795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-0fe9864c-eddf-44e2-9b98-4a66392c6051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd23bf8e26c2f46108d6969e5a43a23e3'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:25.126404 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1281/2587] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:53:24 2022] DELETE /identity/v3/users/d23bf8e26c2f46108d6969e5a43a23e3 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:53:25.131792 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ffa5e-bb1d-4afa-849c-7e6784063838 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:25.132175 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ffa5e-bb1d-4afa-849c-7e6784063838 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:25.132540 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ffa5e-bb1d-4afa-849c-7e6784063838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:25.152502 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-325ffa5e-bb1d-4afa-849c-7e6784063838 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:25.154098 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-325ffa5e-bb1d-4afa-849c-7e6784063838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:25.158507 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:25.161865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbe56c-8af3-463b-b01d-129d5c148251 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:25.198199 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:25.222049 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1282/2588] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:53:25.225167 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806f49c-ba54-44d2-a555-9a01f5a4e896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:25.226076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806f49c-ba54-44d2-a555-9a01f5a4e896 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:25.226766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806f49c-ba54-44d2-a555-9a01f5a4e896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:25.226877 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806f49c-ba54-44d2-a555-9a01f5a4e896 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:25.227523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806f49c-ba54-44d2-a555-9a01f5a4e896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:25.229870 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806f49c-ba54-44d2-a555-9a01f5a4e896 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:25.234268 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ffa5e-bb1d-4afa-849c-7e6784063838 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:25.235719 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-325ffa5e-bb1d-4afa-849c-7e6784063838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:25.297206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ffa5e-bb1d-4afa-849c-7e6784063838 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:25.300948 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40529a35-ef39-422a-9862-337ab85f731c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:25.301794 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40529a35-ef39-422a-9862-337ab85f731c service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:25.302245 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40529a35-ef39-422a-9862-337ab85f731c service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:25.302714 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40529a35-ef39-422a-9862-337ab85f731c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:25.313143 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40529a35-ef39-422a-9862-337ab85f731c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:25.314633 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40529a35-ef39-422a-9862-337ab85f731c service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:25.326497 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1283/2589] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:53:25.366771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1307/2590] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:53:25 2022] POST /identity/v3/projects => generated 385 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:53:25.371108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40529a35-ef39-422a-9862-337ab85f731c service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:25.373692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-40529a35-ef39-422a-9862-337ab85f731c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:25.388408 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40529a35-ef39-422a-9862-337ab85f731c service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:25.451969 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b9c15-9037-4d98-8c6d-be535d0a08f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:25.452798 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b9c15-9037-4d98-8c6d-be535d0a08f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:25.453155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b9c15-9037-4d98-8c6d-be535d0a08f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:25.453508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b9c15-9037-4d98-8c6d-be535d0a08f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:25.454351 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b9c15-9037-4d98-8c6d-be535d0a08f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:25.456324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b9c15-9037-4d98-8c6d-be535d0a08f9 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:25.572735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1284/2591] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:53:25 2022] POST /identity/v3/users => generated 299 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:25.577776 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806f49c-ba54-44d2-a555-9a01f5a4e896 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:25.594304 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806f49c-ba54-44d2-a555-9a01f5a4e896 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:25.601382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2e06a-f640-4123-aa09-9563d2e9a70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:25.602085 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2e06a-f640-4123-aa09-9563d2e9a70c admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:25.602414 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2e06a-f640-4123-aa09-9563d2e9a70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:25.602688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2e06a-f640-4123-aa09-9563d2e9a70c admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:25.603382 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee2e06a-f640-4123-aa09-9563d2e9a70c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:25.605298 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee2e06a-f640-4123-aa09-9563d2e9a70c admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:25.618882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1308/2592] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:25.623032 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b9c15-9037-4d98-8c6d-be535d0a08f9 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:25.635123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b9c15-9037-4d98-8c6d-be535d0a08f9 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:25.641517 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28663ab-e929-4d16-a80d-bb55394b8be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:25.644138 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28663ab-e929-4d16-a80d-bb55394b8be4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:25.644138 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28663ab-e929-4d16-a80d-bb55394b8be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:25.644138 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28663ab-e929-4d16-a80d-bb55394b8be4 admin admin] PATH_INFO: `/v3/projects/13f99f7632f04ce3b29f4a69b8cccbe1/users/ae050adbc8ec4f34b093564dd7580ba0/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:25.667542 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28663ab-e929-4d16-a80d-bb55394b8be4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f99f7632f04ce3b29f4a69b8cccbe1, user_id=ae050adbc8ec4f34b093564dd7580ba0, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:25.670556 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28663ab-e929-4d16-a80d-bb55394b8be4 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:25.757024 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1285/2593] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:53:25 2022] PUT /identity/v3/projects/13f99f7632f04ce3b29f4a69b8cccbe1/users/ae050adbc8ec4f34b093564dd7580ba0/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:53:25.760917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2e06a-f640-4123-aa09-9563d2e9a70c admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:25.772131 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2e06a-f640-4123-aa09-9563d2e9a70c admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:25.790217 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28663ab-e929-4d16-a80d-bb55394b8be4 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:25.792020 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-f28663ab-e929-4d16-a80d-bb55394b8be4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:25.834705 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:25.888181 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428c9fe-6160-4588-a677-e0f4c146ebbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:25.888953 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428c9fe-6160-4588-a677-e0f4c146ebbf admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:25.889243 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428c9fe-6160-4588-a677-e0f4c146ebbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:25.889527 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428c9fe-6160-4588-a677-e0f4c146ebbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:25.890263 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d428c9fe-6160-4588-a677-e0f4c146ebbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:25.891949 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d428c9fe-6160-4588-a677-e0f4c146ebbf admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:25.900316 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28663ab-e929-4d16-a80d-bb55394b8be4 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:25.904863 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1309/2594] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:25.906134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b337e2-f2d6-4ac0-ba79-a37e30369273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:25.907077 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b337e2-f2d6-4ac0-ba79-a37e30369273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:25.907392 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b337e2-f2d6-4ac0-ba79-a37e30369273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:25.907647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b337e2-f2d6-4ac0-ba79-a37e30369273 admin admin] PATH_INFO: `/v3/projects/385c21202aec4dacbb6eac46653bab66` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:25.908703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428c9fe-6160-4588-a677-e0f4c146ebbf admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:25.909085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b337e2-f2d6-4ac0-ba79-a37e30369273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=385c21202aec4dacbb6eac46653bab66)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:25.911058 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b337e2-f2d6-4ac0-ba79-a37e30369273 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:25.919535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428c9fe-6160-4588-a677-e0f4c146ebbf admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:25.924496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439cf4e-d538-4565-9a83-735aa020d752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:25.929176 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439cf4e-d538-4565-9a83-735aa020d752 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:25.929176 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439cf4e-d538-4565-9a83-735aa020d752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:25.929176 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439cf4e-d538-4565-9a83-735aa020d752 admin admin] PATH_INFO: `/v3/projects/13f99f7632f04ce3b29f4a69b8cccbe1/users/ae050adbc8ec4f34b093564dd7580ba0/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:25.929176 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439cf4e-d538-4565-9a83-735aa020d752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f99f7632f04ce3b29f4a69b8cccbe1, user_id=ae050adbc8ec4f34b093564dd7580ba0, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:25.930930 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439cf4e-d538-4565-9a83-735aa020d752 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:25.959014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1310/2595] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:53:25 2022] PUT /identity/v3/projects/13f99f7632f04ce3b29f4a69b8cccbe1/users/ae050adbc8ec4f34b093564dd7580ba0/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:25.959014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254d78e-47fa-4fdb-91e4-bf7710dfdad6 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:25.959014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254d78e-47fa-4fdb-91e4-bf7710dfdad6 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:25.959552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254d78e-47fa-4fdb-91e4-bf7710dfdad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:25.987453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-6254d78e-47fa-4fdb-91e4-bf7710dfdad6 None None] MFA Rules not processed for user `ae050adbc8ec4f34b093564dd7580ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:25.988802 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6254d78e-47fa-4fdb-91e4-bf7710dfdad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:26.035300 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-80b337e2-f2d6-4ac0-ba79-a37e30369273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '385c21202aec4dacbb6eac46653bab66', 'request_id': 'req-80b337e2-f2d6-4ac0-ba79-a37e30369273'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:26.037940 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:26.071249 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1311/2596] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:53:26.084370 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254d78e-47fa-4fdb-91e4-bf7710dfdad6 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:26.087244 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6254d78e-47fa-4fdb-91e4-bf7710dfdad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:26.109261 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-80b337e2-f2d6-4ac0-ba79-a37e30369273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '385c21202aec4dacbb6eac46653bab66', 'request_id': 'req-80b337e2-f2d6-4ac0-ba79-a37e30369273'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:26.116792 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-80b337e2-f2d6-4ac0-ba79-a37e30369273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '385c21202aec4dacbb6eac46653bab66', 'request_id': 'req-80b337e2-f2d6-4ac0-ba79-a37e30369273'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:26.129351 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-80b337e2-f2d6-4ac0-ba79-a37e30369273 admin admin] The token cache is being invalidate because project 385c21202aec4dacbb6eac46653bab66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:53:26.129616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-80b337e2-f2d6-4ac0-ba79-a37e30369273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:26.131191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-80b337e2-f2d6-4ac0-ba79-a37e30369273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:26.133015 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1286/2597] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:53:25 2022] DELETE /identity/v3/projects/385c21202aec4dacbb6eac46653bab66 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:53:26.136980 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b337e2-f2d6-4ac0-ba79-a37e30369273 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:26.138192 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-80b337e2-f2d6-4ac0-ba79-a37e30369273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:26.157324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254d78e-47fa-4fdb-91e4-bf7710dfdad6 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:26.216066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b337e2-f2d6-4ac0-ba79-a37e30369273 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:26.217233 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe66283-76d4-42d3-bed0-8339cad3add4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:26.217904 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe66283-76d4-42d3-bed0-8339cad3add4 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:26.218192 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe66283-76d4-42d3-bed0-8339cad3add4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:26.218494 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe66283-76d4-42d3-bed0-8339cad3add4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:26.220299 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-cbe66283-76d4-42d3-bed0-8339cad3add4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:26.222434 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf689a-1f05-4e9f-b3cc-fd54b2848659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:26.223530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf689a-1f05-4e9f-b3cc-fd54b2848659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:26.224147 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf689a-1f05-4e9f-b3cc-fd54b2848659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:26.224607 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf689a-1f05-4e9f-b3cc-fd54b2848659 admin admin] PATH_INFO: `/v3/users/ec6fda27a78045a8bfbb58ebbcede225` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:26.226249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf689a-1f05-4e9f-b3cc-fd54b2848659 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec6fda27a78045a8bfbb58ebbcede225)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:26.229849 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf689a-1f05-4e9f-b3cc-fd54b2848659 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:26.299830 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe66283-76d4-42d3-bed0-8339cad3add4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:26.301584 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe66283-76d4-42d3-bed0-8339cad3add4 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:26.313000 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1312/2598] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:53:26.432621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-41cf689a-1f05-4e9f-b3cc-fd54b2848659 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec6fda27a78045a8bfbb58ebbcede225'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:26.435687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:26.453104 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-41cf689a-1f05-4e9f-b3cc-fd54b2848659 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec6fda27a78045a8bfbb58ebbcede225'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:26.546012 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-41cf689a-1f05-4e9f-b3cc-fd54b2848659 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec6fda27a78045a8bfbb58ebbcede225'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:26.546807 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-41cf689a-1f05-4e9f-b3cc-fd54b2848659 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec6fda27a78045a8bfbb58ebbcede225'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:26.547495 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-41cf689a-1f05-4e9f-b3cc-fd54b2848659 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec6fda27a78045a8bfbb58ebbcede225'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:26.561682 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1287/2599] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:53:26 2022] DELETE /identity/v3/users/ec6fda27a78045a8bfbb58ebbcede225 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:27.143703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe66283-76d4-42d3-bed0-8339cad3add4 service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:27.169124 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe66283-76d4-42d3-bed0-8339cad3add4 service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:27.255550 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:27.270976 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21836245-164c-4926-85cc-31a783f6201b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:27.271803 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21836245-164c-4926-85cc-31a783f6201b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:27.272304 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21836245-164c-4926-85cc-31a783f6201b admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:27.272566 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21836245-164c-4926-85cc-31a783f6201b admin admin] PATH_INFO: `/v3/projects/668c9933534a45f5a216e34d34717952` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:27.273962 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21836245-164c-4926-85cc-31a783f6201b admin admin] RBAC: Authorizing `identity:delete_project(project_id=668c9933534a45f5a216e34d34717952)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:27.275924 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21836245-164c-4926-85cc-31a783f6201b admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:27.394349 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-21836245-164c-4926-85cc-31a783f6201b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '668c9933534a45f5a216e34d34717952', 'request_id': 'req-21836245-164c-4926-85cc-31a783f6201b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:27.402095 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-21836245-164c-4926-85cc-31a783f6201b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '668c9933534a45f5a216e34d34717952', 'request_id': 'req-21836245-164c-4926-85cc-31a783f6201b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:27.408049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-21836245-164c-4926-85cc-31a783f6201b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '668c9933534a45f5a216e34d34717952', 'request_id': 'req-21836245-164c-4926-85cc-31a783f6201b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:27.500551 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-21836245-164c-4926-85cc-31a783f6201b admin admin] The token cache is being invalidate because project 668c9933534a45f5a216e34d34717952 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:53:27.500880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-21836245-164c-4926-85cc-31a783f6201b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:27.501548 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-21836245-164c-4926-85cc-31a783f6201b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:27.505015 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1313/2600] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:53:27 2022] DELETE /identity/v3/projects/668c9933534a45f5a216e34d34717952 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:53:27.508484 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ec132-44e1-4fd2-b9bb-4c6c734fdaa6 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:27.509034 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ec132-44e1-4fd2-b9bb-4c6c734fdaa6 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:27.509123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ec132-44e1-4fd2-b9bb-4c6c734fdaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:27.528712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-ed0ec132-44e1-4fd2-b9bb-4c6c734fdaa6 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:27.529928 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ed0ec132-44e1-4fd2-b9bb-4c6c734fdaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:27.595281 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1288/2601] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:53:27.605687 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2a0ae-522a-4299-92d0-4b4a9ee4033d None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:27.605986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2a0ae-522a-4299-92d0-4b4a9ee4033d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:27.606270 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2a0ae-522a-4299-92d0-4b4a9ee4033d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:27.626168 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-3ea2a0ae-522a-4299-92d0-4b4a9ee4033d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:27.629426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3ea2a0ae-522a-4299-92d0-4b4a9ee4033d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:27.635496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1314/2602] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:27.639812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ec132-44e1-4fd2-b9bb-4c6c734fdaa6 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:27.650695 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ec132-44e1-4fd2-b9bb-4c6c734fdaa6 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:27.655818 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feaab8c-c09f-4f96-9f9d-735d4439ad5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:27.656485 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feaab8c-c09f-4f96-9f9d-735d4439ad5e admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:27.656812 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feaab8c-c09f-4f96-9f9d-735d4439ad5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:27.657094 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feaab8c-c09f-4f96-9f9d-735d4439ad5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:27.657826 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feaab8c-c09f-4f96-9f9d-735d4439ad5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:53:27.658139 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feaab8c-c09f-4f96-9f9d-735d4439ad5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:27.659802 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feaab8c-c09f-4f96-9f9d-735d4439ad5e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:27.672530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1289/2603] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:53:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:53:27.677811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2a0ae-522a-4299-92d0-4b4a9ee4033d None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:27.690901 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2a0ae-522a-4299-92d0-4b4a9ee4033d None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:27.696441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab459e-6c14-413d-835e-3385efcdd0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:27.697173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab459e-6c14-413d-835e-3385efcdd0eb admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:27.697469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab459e-6c14-413d-835e-3385efcdd0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:27.697745 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab459e-6c14-413d-835e-3385efcdd0eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:27.698618 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab459e-6c14-413d-835e-3385efcdd0eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:27.701994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab459e-6c14-413d-835e-3385efcdd0eb admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:27.815853 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1315/2604] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:53:27 2022] POST /identity/v3/projects => generated 389 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:27.817063 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feaab8c-c09f-4f96-9f9d-735d4439ad5e admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:27.828044 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feaab8c-c09f-4f96-9f9d-735d4439ad5e admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:27.895161 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2f590-822f-49c3-a805-2cb30c8cc9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:27.896612 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2f590-822f-49c3-a805-2cb30c8cc9d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:27.897358 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2f590-822f-49c3-a805-2cb30c8cc9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:27.897358 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2f590-822f-49c3-a805-2cb30c8cc9d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:27.897861 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2f590-822f-49c3-a805-2cb30c8cc9d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:27.900249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2f590-822f-49c3-a805-2cb30c8cc9d9 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:27.969065 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1290/2605] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:53:27 2022] POST /identity/v3/users => generated 295 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:27.973350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab459e-6c14-413d-835e-3385efcdd0eb admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:27.986795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab459e-6c14-413d-835e-3385efcdd0eb admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:27.991780 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6ecc7-48c7-49fc-92aa-a946da134336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:27.992443 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6ecc7-48c7-49fc-92aa-a946da134336 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:27.992758 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6ecc7-48c7-49fc-92aa-a946da134336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:27.993039 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6ecc7-48c7-49fc-92aa-a946da134336 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:27.993768 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6ecc7-48c7-49fc-92aa-a946da134336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:27.995392 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6ecc7-48c7-49fc-92aa-a946da134336 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:28.011235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1316/2606] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:28.011341 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2f590-822f-49c3-a805-2cb30c8cc9d9 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:28.030823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2f590-822f-49c3-a805-2cb30c8cc9d9 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:28.035788 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83370e7e-fc1d-4f4a-adf9-99842b8b1f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:28.037271 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83370e7e-fc1d-4f4a-adf9-99842b8b1f0b admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:28.042126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83370e7e-fc1d-4f4a-adf9-99842b8b1f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:28.042126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83370e7e-fc1d-4f4a-adf9-99842b8b1f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:28.042126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83370e7e-fc1d-4f4a-adf9-99842b8b1f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:28.042126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83370e7e-fc1d-4f4a-adf9-99842b8b1f0b admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:28.051679 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1291/2607] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:53:28.056038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6ecc7-48c7-49fc-92aa-a946da134336 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:28.066131 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6ecc7-48c7-49fc-92aa-a946da134336 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:28.071016 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea4a81-de35-40ef-b138-4e2b74bbd606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:28.071886 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea4a81-de35-40ef-b138-4e2b74bbd606 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:28.072169 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea4a81-de35-40ef-b138-4e2b74bbd606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:28.073218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea4a81-de35-40ef-b138-4e2b74bbd606 admin admin] PATH_INFO: `/v3/projects/1796c3c9a0b149f9842a8562c4836964/users/47207978decc4c5f98c040621ff31d60/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:28.089374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea4a81-de35-40ef-b138-4e2b74bbd606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1796c3c9a0b149f9842a8562c4836964, user_id=47207978decc4c5f98c040621ff31d60, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:28.092068 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea4a81-de35-40ef-b138-4e2b74bbd606 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:28.205661 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1317/2608] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:53:28 2022] PUT /identity/v3/projects/1796c3c9a0b149f9842a8562c4836964/users/47207978decc4c5f98c040621ff31d60/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:28.214976 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6805850-f38c-4619-bb01-48b49b46243a None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:28.215269 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6805850-f38c-4619-bb01-48b49b46243a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:28.215546 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6805850-f38c-4619-bb01-48b49b46243a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:28.263011 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-a6805850-f38c-4619-bb01-48b49b46243a None None] MFA Rules not processed for user `47207978decc4c5f98c040621ff31d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:28.264247 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-a6805850-f38c-4619-bb01-48b49b46243a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:28.299479 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:28.355241 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1292/2609] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:28.360564 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea4a81-de35-40ef-b138-4e2b74bbd606 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:28.430689 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6805850-f38c-4619-bb01-48b49b46243a None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:28.434365 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-a6805850-f38c-4619-bb01-48b49b46243a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:28.448166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea4a81-de35-40ef-b138-4e2b74bbd606 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:28.455610 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77c046-c4a5-4dc3-8b6a-9988be6e5f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:28.456285 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77c046-c4a5-4dc3-8b6a-9988be6e5f1e admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:28.456636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77c046-c4a5-4dc3-8b6a-9988be6e5f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:28.456920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77c046-c4a5-4dc3-8b6a-9988be6e5f1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:28.457673 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77c046-c4a5-4dc3-8b6a-9988be6e5f1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:28.462208 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77c046-c4a5-4dc3-8b6a-9988be6e5f1e admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:28.513949 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6805850-f38c-4619-bb01-48b49b46243a None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:28.517658 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5088c2d-a85f-4598-95c3-c31a31549872 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:28.518311 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5088c2d-a85f-4598-95c3-c31a31549872 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:28.518588 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5088c2d-a85f-4598-95c3-c31a31549872 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:28.518864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5088c2d-a85f-4598-95c3-c31a31549872 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:28.521154 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c5088c2d-a85f-4598-95c3-c31a31549872 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:28.551736 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:28.566325 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1318/2610] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:53:28 2022] POST /identity/v3/projects => generated 387 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:28.567447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77c046-c4a5-4dc3-8b6a-9988be6e5f1e admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:28.585076 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5088c2d-a85f-4598-95c3-c31a31549872 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:28.586545 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5088c2d-a85f-4598-95c3-c31a31549872 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:28.597322 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1293/2611] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:53:28.650785 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77c046-c4a5-4dc3-8b6a-9988be6e5f1e admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:28.655626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d857e0c-0d06-466f-9b2e-92554d1f1af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:28.656307 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d857e0c-0d06-466f-9b2e-92554d1f1af1 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:28.657226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d857e0c-0d06-466f-9b2e-92554d1f1af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:28.657226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d857e0c-0d06-466f-9b2e-92554d1f1af1 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:28.657594 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d857e0c-0d06-466f-9b2e-92554d1f1af1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:28.659544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d857e0c-0d06-466f-9b2e-92554d1f1af1 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:28.764448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5088c2d-a85f-4598-95c3-c31a31549872 service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:28.764448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c5088c2d-a85f-4598-95c3-c31a31549872 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:28.774481 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5088c2d-a85f-4598-95c3-c31a31549872 service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:28.779198 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a801808-2bb8-4310-ad55-de0777ca589e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:28.780070 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a801808-2bb8-4310-ad55-de0777ca589e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:28.780359 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a801808-2bb8-4310-ad55-de0777ca589e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:28.780803 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a801808-2bb8-4310-ad55-de0777ca589e admin admin] PATH_INFO: `/v3/users/e6a2de8d5a514a73aa4e136b7afee9c2` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:28.781903 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a801808-2bb8-4310-ad55-de0777ca589e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6a2de8d5a514a73aa4e136b7afee9c2)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:28.784160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a801808-2bb8-4310-ad55-de0777ca589e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:28.789831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1319/2612] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:53:28 2022] POST /identity/v3/users => generated 300 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:28.797191 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d857e0c-0d06-466f-9b2e-92554d1f1af1 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:28.809010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d857e0c-0d06-466f-9b2e-92554d1f1af1 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:28.815239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bf084-4196-4c0f-b2eb-86385cc638fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:28.815916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bf084-4196-4c0f-b2eb-86385cc638fd admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:28.816205 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bf084-4196-4c0f-b2eb-86385cc638fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:28.817162 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bf084-4196-4c0f-b2eb-86385cc638fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:28.818159 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bf084-4196-4c0f-b2eb-86385cc638fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:28.828992 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bf084-4196-4c0f-b2eb-86385cc638fd admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:28.831058 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1320/2613] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:53:28.835790 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bf084-4196-4c0f-b2eb-86385cc638fd admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:28.845808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bf084-4196-4c0f-b2eb-86385cc638fd admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:28.851585 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9cb02b-d4c4-4189-aca1-66a6ad88a5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:28.852501 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cb02b-d4c4-4189-aca1-66a6ad88a5d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:28.852811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cb02b-d4c4-4189-aca1-66a6ad88a5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:28.853088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cb02b-d4c4-4189-aca1-66a6ad88a5d9 admin admin] PATH_INFO: `/v3/projects/c9b31c6a6f264dfc82455c09dc3e375b/users/6436eaaa8af1436bb90f24581a637a70/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:28.869003 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9cb02b-d4c4-4189-aca1-66a6ad88a5d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b31c6a6f264dfc82455c09dc3e375b, user_id=6436eaaa8af1436bb90f24581a637a70, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:28.871662 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9cb02b-d4c4-4189-aca1-66a6ad88a5d9 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:28.967622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1321/2614] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:53:28 2022] PUT /identity/v3/projects/c9b31c6a6f264dfc82455c09dc3e375b/users/6436eaaa8af1436bb90f24581a637a70/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:53:28.971918 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9cb02b-d4c4-4189-aca1-66a6ad88a5d9 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:29.049176 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9cb02b-d4c4-4189-aca1-66a6ad88a5d9 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:29.055441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7059af4-353e-4fa6-8915-283825dd8b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:29.056313 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7059af4-353e-4fa6-8915-283825dd8b0b admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:29.056617 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7059af4-353e-4fa6-8915-283825dd8b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:29.056942 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7059af4-353e-4fa6-8915-283825dd8b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:29.058324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7059af4-353e-4fa6-8915-283825dd8b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:29.073571 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7059af4-353e-4fa6-8915-283825dd8b0b admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:29.073571 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1322/2615] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:53:29.080795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7059af4-353e-4fa6-8915-283825dd8b0b admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:29.092277 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7059af4-353e-4fa6-8915-283825dd8b0b admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:29.097211 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf7113-9e84-4bfb-a287-6a3af527b15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:29.098080 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf7113-9e84-4bfb-a287-6a3af527b15b admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:29.098367 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf7113-9e84-4bfb-a287-6a3af527b15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:29.098763 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf7113-9e84-4bfb-a287-6a3af527b15b admin admin] PATH_INFO: `/v3/projects/c9b31c6a6f264dfc82455c09dc3e375b/users/6436eaaa8af1436bb90f24581a637a70/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:29.101214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cf7113-9e84-4bfb-a287-6a3af527b15b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b31c6a6f264dfc82455c09dc3e375b, user_id=6436eaaa8af1436bb90f24581a637a70, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:29.104106 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cf7113-9e84-4bfb-a287-6a3af527b15b admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:29.114158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1323/2616] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:53:29 2022] PUT /identity/v3/projects/c9b31c6a6f264dfc82455c09dc3e375b/users/6436eaaa8af1436bb90f24581a637a70/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:29.124112 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3d490-7bd6-469f-af1d-50416e05fcde None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:29.124408 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3d490-7bd6-469f-af1d-50416e05fcde None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:29.124720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3d490-7bd6-469f-af1d-50416e05fcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:29.151362 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-33e3d490-7bd6-469f-af1d-50416e05fcde None None] MFA Rules not processed for user `6436eaaa8af1436bb90f24581a637a70`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:29.152569 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-33e3d490-7bd6-469f-af1d-50416e05fcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:29.175242 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-0a801808-2bb8-4310-ad55-de0777ca589e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6a2de8d5a514a73aa4e136b7afee9c2'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:29.178334 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:29.188220 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-0a801808-2bb8-4310-ad55-de0777ca589e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6a2de8d5a514a73aa4e136b7afee9c2'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:29.221987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1324/2617] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:53:29.233206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3d490-7bd6-469f-af1d-50416e05fcde None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:29.234615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-33e3d490-7bd6-469f-af1d-50416e05fcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:29.283535 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-0a801808-2bb8-4310-ad55-de0777ca589e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6a2de8d5a514a73aa4e136b7afee9c2'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:29.288719 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-0a801808-2bb8-4310-ad55-de0777ca589e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6a2de8d5a514a73aa4e136b7afee9c2'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:29.292174 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-0a801808-2bb8-4310-ad55-de0777ca589e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6a2de8d5a514a73aa4e136b7afee9c2'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:29.292174 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:29.297771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3d490-7bd6-469f-af1d-50416e05fcde None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:29.301453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e0777-8431-4a9a-83ee-d69b9834e656 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:29.302673 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e0777-8431-4a9a-83ee-d69b9834e656 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:29.302981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e0777-8431-4a9a-83ee-d69b9834e656 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:29.303266 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e0777-8431-4a9a-83ee-d69b9834e656 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:29.304231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1294/2618] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:53:28 2022] DELETE /identity/v3/users/e6a2de8d5a514a73aa4e136b7afee9c2 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:29.308157 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-183e0777-8431-4a9a-83ee-d69b9834e656 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:29.322920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e0777-8431-4a9a-83ee-d69b9834e656 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:29.324146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e0777-8431-4a9a-83ee-d69b9834e656 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:29.334336 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1325/2619] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:53:29.469602 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a801808-2bb8-4310-ad55-de0777ca589e admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:29.471302 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-0a801808-2bb8-4310-ad55-de0777ca589e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:29.536868 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a801808-2bb8-4310-ad55-de0777ca589e admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:29.541750 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e321d3-ef71-44df-95bb-4bb9da607fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:29.542623 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e321d3-ef71-44df-95bb-4bb9da607fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:29.542908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e321d3-ef71-44df-95bb-4bb9da607fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:29.543184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e321d3-ef71-44df-95bb-4bb9da607fd1 admin admin] PATH_INFO: `/v3/users/47207978decc4c5f98c040621ff31d60` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:29.544438 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e321d3-ef71-44df-95bb-4bb9da607fd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47207978decc4c5f98c040621ff31d60)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:29.546420 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e321d3-ef71-44df-95bb-4bb9da607fd1 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:29.591172 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e0777-8431-4a9a-83ee-d69b9834e656 service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:29.592602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-183e0777-8431-4a9a-83ee-d69b9834e656 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:29.662062 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e0777-8431-4a9a-83ee-d69b9834e656 service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:29.666780 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c65dca-8280-45c8-848a-e37a3ca52f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:29.667526 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c65dca-8280-45c8-848a-e37a3ca52f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:29.667813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c65dca-8280-45c8-848a-e37a3ca52f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:29.668107 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c65dca-8280-45c8-848a-e37a3ca52f36 admin admin] PATH_INFO: `/v3/projects/e3f62d8ed71b424f8414e9c3eaa681aa` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:29.669324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c65dca-8280-45c8-848a-e37a3ca52f36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3f62d8ed71b424f8414e9c3eaa681aa)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:29.671272 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c65dca-8280-45c8-848a-e37a3ca52f36 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:29.746079 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d9e321d3-ef71-44df-95bb-4bb9da607fd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47207978decc4c5f98c040621ff31d60'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:29.758234 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d9e321d3-ef71-44df-95bb-4bb9da607fd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47207978decc4c5f98c040621ff31d60'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:29.781242 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-12c65dca-8280-45c8-848a-e37a3ca52f36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3f62d8ed71b424f8414e9c3eaa681aa', 'request_id': 'req-12c65dca-8280-45c8-848a-e37a3ca52f36'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:29.787831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-12c65dca-8280-45c8-848a-e37a3ca52f36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3f62d8ed71b424f8414e9c3eaa681aa', 'request_id': 'req-12c65dca-8280-45c8-848a-e37a3ca52f36'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:29.793538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-12c65dca-8280-45c8-848a-e37a3ca52f36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3f62d8ed71b424f8414e9c3eaa681aa', 'request_id': 'req-12c65dca-8280-45c8-848a-e37a3ca52f36'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:29.798223 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:29.881542 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d9e321d3-ef71-44df-95bb-4bb9da607fd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47207978decc4c5f98c040621ff31d60'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:29.882083 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d9e321d3-ef71-44df-95bb-4bb9da607fd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47207978decc4c5f98c040621ff31d60'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:29.882686 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d9e321d3-ef71-44df-95bb-4bb9da607fd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47207978decc4c5f98c040621ff31d60'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:29.897015 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1295/2620] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:53:29 2022] DELETE /identity/v3/users/47207978decc4c5f98c040621ff31d60 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:29.901438 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ce2e5-0f30-4469-b40d-0c31c7254297 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:29.901733 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ce2e5-0f30-4469-b40d-0c31c7254297 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:29.902017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ce2e5-0f30-4469-b40d-0c31c7254297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:29.919616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-740ce2e5-0f30-4469-b40d-0c31c7254297 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:29.920865 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-740ce2e5-0f30-4469-b40d-0c31c7254297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:29.954184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-12c65dca-8280-45c8-848a-e37a3ca52f36 admin admin] The token cache is being invalidate because project e3f62d8ed71b424f8414e9c3eaa681aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:53:29.955579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-12c65dca-8280-45c8-848a-e37a3ca52f36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:29.955579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-12c65dca-8280-45c8-848a-e37a3ca52f36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:29.956923 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1326/2621] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:53:29 2022] DELETE /identity/v3/projects/e3f62d8ed71b424f8414e9c3eaa681aa => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:53:29.961329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c65dca-8280-45c8-848a-e37a3ca52f36 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:29.962708 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-12c65dca-8280-45c8-848a-e37a3ca52f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:29.982797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1296/2622] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:53:29.992533 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ce2e5-0f30-4469-b40d-0c31c7254297 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:29.994384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-740ce2e5-0f30-4469-b40d-0c31c7254297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:30.005856 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:30.034904 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c65dca-8280-45c8-848a-e37a3ca52f36 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:30.040090 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b229e-ea4d-4fdb-97dd-25b093f4fa36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:30.041009 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b229e-ea4d-4fdb-97dd-25b093f4fa36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:30.041502 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b229e-ea4d-4fdb-97dd-25b093f4fa36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:30.041719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b229e-ea4d-4fdb-97dd-25b093f4fa36 admin admin] PATH_INFO: `/v3/users/498c2d8bc3f1481099c60293525cb33f` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:30.042842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b229e-ea4d-4fdb-97dd-25b093f4fa36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=498c2d8bc3f1481099c60293525cb33f)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:30.044791 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b229e-ea4d-4fdb-97dd-25b093f4fa36 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:30.053076 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ce2e5-0f30-4469-b40d-0c31c7254297 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:30.057077 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd36d7c-eb39-40c1-8e6c-2e03700d7ec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:30.057862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd36d7c-eb39-40c1-8e6c-2e03700d7ec1 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:30.058270 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd36d7c-eb39-40c1-8e6c-2e03700d7ec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:30.058666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd36d7c-eb39-40c1-8e6c-2e03700d7ec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:30.069219 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd36d7c-eb39-40c1-8e6c-2e03700d7ec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:30.070671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd36d7c-eb39-40c1-8e6c-2e03700d7ec1 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:30.080223 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1297/2623] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:53:30.243732 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c95b229e-ea4d-4fdb-97dd-25b093f4fa36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '498c2d8bc3f1481099c60293525cb33f'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:30.261913 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c95b229e-ea4d-4fdb-97dd-25b093f4fa36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '498c2d8bc3f1481099c60293525cb33f'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:30.261913 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c95b229e-ea4d-4fdb-97dd-25b093f4fa36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '498c2d8bc3f1481099c60293525cb33f'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:30.261913 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c95b229e-ea4d-4fdb-97dd-25b093f4fa36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '498c2d8bc3f1481099c60293525cb33f'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:30.267296 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c95b229e-ea4d-4fdb-97dd-25b093f4fa36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '498c2d8bc3f1481099c60293525cb33f'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:30.335050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd36d7c-eb39-40c1-8e6c-2e03700d7ec1 service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:30.337586 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-1cd36d7c-eb39-40c1-8e6c-2e03700d7ec1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:30.373216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1327/2624] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:53:29 2022] DELETE /identity/v3/users/498c2d8bc3f1481099c60293525cb33f => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:53:30.398486 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd36d7c-eb39-40c1-8e6c-2e03700d7ec1 service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:30.454315 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1417a-93c0-4614-b5bf-8d55063ffadc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:30.455101 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1417a-93c0-4614-b5bf-8d55063ffadc service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:30.455388 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1417a-93c0-4614-b5bf-8d55063ffadc service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:30.455671 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1417a-93c0-4614-b5bf-8d55063ffadc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:30.457525 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-67b1417a-93c0-4614-b5bf-8d55063ffadc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:30.521940 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1417a-93c0-4614-b5bf-8d55063ffadc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:30.523390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1417a-93c0-4614-b5bf-8d55063ffadc service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:30.533489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1298/2625] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:53:30.545586 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b229e-ea4d-4fdb-97dd-25b093f4fa36 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:30.552461 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-c95b229e-ea4d-4fdb-97dd-25b093f4fa36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:30.619697 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b229e-ea4d-4fdb-97dd-25b093f4fa36 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:30.624282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119ef1d-06a6-455c-bd96-f6073bd6d42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:30.625171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119ef1d-06a6-455c-bd96-f6073bd6d42c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:30.625460 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119ef1d-06a6-455c-bd96-f6073bd6d42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:30.625739 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119ef1d-06a6-455c-bd96-f6073bd6d42c admin admin] PATH_INFO: `/v3/projects/1796c3c9a0b149f9842a8562c4836964` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:30.626919 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119ef1d-06a6-455c-bd96-f6073bd6d42c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1796c3c9a0b149f9842a8562c4836964)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:30.628864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119ef1d-06a6-455c-bd96-f6073bd6d42c admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:30.638324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1417a-93c0-4614-b5bf-8d55063ffadc service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:30.639740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-67b1417a-93c0-4614-b5bf-8d55063ffadc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:30.649717 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:30.667461 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1417a-93c0-4614-b5bf-8d55063ffadc service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:30.671862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123d3ad-c7bc-4602-bb4c-c91c7995f983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:30.672734 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123d3ad-c7bc-4602-bb4c-c91c7995f983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:30.673086 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123d3ad-c7bc-4602-bb4c-c91c7995f983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:30.673361 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123d3ad-c7bc-4602-bb4c-c91c7995f983 admin admin] PATH_INFO: `/v3/projects/17d5c759351445b6aeaf4f31685f321b` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:30.674530 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123d3ad-c7bc-4602-bb4c-c91c7995f983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17d5c759351445b6aeaf4f31685f321b)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:30.676631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123d3ad-c7bc-4602-bb4c-c91c7995f983 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:30.688720 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:30.801049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-7119ef1d-06a6-455c-bd96-f6073bd6d42c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1796c3c9a0b149f9842a8562c4836964', 'request_id': 'req-7119ef1d-06a6-455c-bd96-f6073bd6d42c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:30.802524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d123d3ad-c7bc-4602-bb4c-c91c7995f983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17d5c759351445b6aeaf4f31685f321b', 'request_id': 'req-d123d3ad-c7bc-4602-bb4c-c91c7995f983'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:30.808983 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-7119ef1d-06a6-455c-bd96-f6073bd6d42c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1796c3c9a0b149f9842a8562c4836964', 'request_id': 'req-7119ef1d-06a6-455c-bd96-f6073bd6d42c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:30.822606 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-7119ef1d-06a6-455c-bd96-f6073bd6d42c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1796c3c9a0b149f9842a8562c4836964', 'request_id': 'req-7119ef1d-06a6-455c-bd96-f6073bd6d42c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:30.888655 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d123d3ad-c7bc-4602-bb4c-c91c7995f983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17d5c759351445b6aeaf4f31685f321b', 'request_id': 'req-d123d3ad-c7bc-4602-bb4c-c91c7995f983'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:30.896026 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d123d3ad-c7bc-4602-bb4c-c91c7995f983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17d5c759351445b6aeaf4f31685f321b', 'request_id': 'req-d123d3ad-c7bc-4602-bb4c-c91c7995f983'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:30.906760 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d123d3ad-c7bc-4602-bb4c-c91c7995f983 admin admin] The token cache is being invalidate because project 17d5c759351445b6aeaf4f31685f321b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:53:30.907052 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d123d3ad-c7bc-4602-bb4c-c91c7995f983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:30.907688 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d123d3ad-c7bc-4602-bb4c-c91c7995f983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:30.909349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1299/2626] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:53:30 2022] DELETE /identity/v3/projects/17d5c759351445b6aeaf4f31685f321b => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:53:30.958360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-7119ef1d-06a6-455c-bd96-f6073bd6d42c admin admin] The token cache is being invalidate because project 1796c3c9a0b149f9842a8562c4836964 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:53:30.958718 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-7119ef1d-06a6-455c-bd96-f6073bd6d42c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:30.959327 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-7119ef1d-06a6-455c-bd96-f6073bd6d42c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:30.960944 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1328/2627] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:53:30 2022] DELETE /identity/v3/projects/1796c3c9a0b149f9842a8562c4836964 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:30.965006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123d3ad-c7bc-4602-bb4c-c91c7995f983 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:30.966425 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-d123d3ad-c7bc-4602-bb4c-c91c7995f983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:31.045044 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123d3ad-c7bc-4602-bb4c-c91c7995f983 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:31.050055 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b7372-8d3f-4acc-a699-0ca1b6192bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:31.050956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b7372-8d3f-4acc-a699-0ca1b6192bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:31.051238 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b7372-8d3f-4acc-a699-0ca1b6192bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:31.051537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b7372-8d3f-4acc-a699-0ca1b6192bd1 admin admin] PATH_INFO: `/v3/users/6436eaaa8af1436bb90f24581a637a70` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:31.053167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b7372-8d3f-4acc-a699-0ca1b6192bd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6436eaaa8af1436bb90f24581a637a70)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:31.055155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b7372-8d3f-4acc-a699-0ca1b6192bd1 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:31.302617 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a32b7372-8d3f-4acc-a699-0ca1b6192bd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6436eaaa8af1436bb90f24581a637a70'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:31.314386 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a32b7372-8d3f-4acc-a699-0ca1b6192bd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6436eaaa8af1436bb90f24581a637a70'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:31.414246 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a32b7372-8d3f-4acc-a699-0ca1b6192bd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6436eaaa8af1436bb90f24581a637a70'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:31.414871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a32b7372-8d3f-4acc-a699-0ca1b6192bd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6436eaaa8af1436bb90f24581a637a70'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:31.415482 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a32b7372-8d3f-4acc-a699-0ca1b6192bd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6436eaaa8af1436bb90f24581a637a70'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:31.428884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1300/2628] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:53:30 2022] DELETE /identity/v3/users/6436eaaa8af1436bb90f24581a637a70 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:31.830343 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119ef1d-06a6-455c-bd96-f6073bd6d42c admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:31.831770 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-7119ef1d-06a6-455c-bd96-f6073bd6d42c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:31.905972 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119ef1d-06a6-455c-bd96-f6073bd6d42c admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:31.967254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2852b24-91a1-48a2-9ded-e0f8a215ce59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:31.968146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2852b24-91a1-48a2-9ded-e0f8a215ce59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:31.968432 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2852b24-91a1-48a2-9ded-e0f8a215ce59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:31.968745 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2852b24-91a1-48a2-9ded-e0f8a215ce59 admin admin] PATH_INFO: `/v3/projects/c9b31c6a6f264dfc82455c09dc3e375b` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:31.969968 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2852b24-91a1-48a2-9ded-e0f8a215ce59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9b31c6a6f264dfc82455c09dc3e375b)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:31.971875 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2852b24-91a1-48a2-9ded-e0f8a215ce59 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:32.164515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b2852b24-91a1-48a2-9ded-e0f8a215ce59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9b31c6a6f264dfc82455c09dc3e375b', 'request_id': 'req-b2852b24-91a1-48a2-9ded-e0f8a215ce59'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:32.170709 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b2852b24-91a1-48a2-9ded-e0f8a215ce59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9b31c6a6f264dfc82455c09dc3e375b', 'request_id': 'req-b2852b24-91a1-48a2-9ded-e0f8a215ce59'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:32.171915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:32.176807 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b2852b24-91a1-48a2-9ded-e0f8a215ce59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9b31c6a6f264dfc82455c09dc3e375b', 'request_id': 'req-b2852b24-91a1-48a2-9ded-e0f8a215ce59'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:32.288960 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b2852b24-91a1-48a2-9ded-e0f8a215ce59 admin admin] The token cache is being invalidate because project c9b31c6a6f264dfc82455c09dc3e375b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:53:32.292031 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b2852b24-91a1-48a2-9ded-e0f8a215ce59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:32.292031 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-b2852b24-91a1-48a2-9ded-e0f8a215ce59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:32.292031 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1329/2629] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:53:31 2022] DELETE /identity/v3/projects/c9b31c6a6f264dfc82455c09dc3e375b => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:32.296455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc678b-25df-4ff0-9d5c-82e284d57939 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:32.296920 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc678b-25df-4ff0-9d5c-82e284d57939 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:32.297074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc678b-25df-4ff0-9d5c-82e284d57939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:32.316340 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-4fcc678b-25df-4ff0-9d5c-82e284d57939 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:32.317655 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-4fcc678b-25df-4ff0-9d5c-82e284d57939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:32.331715 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:32.378884 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1301/2630] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:32.387249 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0aea2-e786-44b5-852e-d3992be66cb6 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:32.387550 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0aea2-e786-44b5-852e-d3992be66cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:32.387836 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0aea2-e786-44b5-852e-d3992be66cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:32.406395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-d0b0aea2-e786-44b5-852e-d3992be66cb6 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:32.407577 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d0b0aea2-e786-44b5-852e-d3992be66cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:32.415827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1330/2631] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:32.420050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc678b-25df-4ff0-9d5c-82e284d57939 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:32.430598 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc678b-25df-4ff0-9d5c-82e284d57939 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:32.435339 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a00d4f-4ed7-4125-a70b-0dfbecbf8c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:32.436003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a00d4f-4ed7-4125-a70b-0dfbecbf8c4e admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:32.436322 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a00d4f-4ed7-4125-a70b-0dfbecbf8c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:32.436618 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a00d4f-4ed7-4125-a70b-0dfbecbf8c4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:32.437351 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a00d4f-4ed7-4125-a70b-0dfbecbf8c4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:53:32.437663 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a00d4f-4ed7-4125-a70b-0dfbecbf8c4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:32.439325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a00d4f-4ed7-4125-a70b-0dfbecbf8c4e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:32.451359 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1302/2632] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:53:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:53:32.455599 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0aea2-e786-44b5-852e-d3992be66cb6 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:32.464845 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0aea2-e786-44b5-852e-d3992be66cb6 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:32.469481 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7787d-d3ca-4468-9170-1aad95426858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:32.470130 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7787d-d3ca-4468-9170-1aad95426858 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:32.470417 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7787d-d3ca-4468-9170-1aad95426858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:32.470697 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7787d-d3ca-4468-9170-1aad95426858 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:32.471421 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7787d-d3ca-4468-9170-1aad95426858 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:32.473374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7787d-d3ca-4468-9170-1aad95426858 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:32.594133 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1331/2633] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:53:32 2022] POST /identity/v3/projects => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:32.598164 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a00d4f-4ed7-4125-a70b-0dfbecbf8c4e admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:32.609409 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a00d4f-4ed7-4125-a70b-0dfbecbf8c4e admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:32.666227 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffa147-0a6d-4dda-b23b-ef15093607c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:32.666805 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffa147-0a6d-4dda-b23b-ef15093607c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:32.667092 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffa147-0a6d-4dda-b23b-ef15093607c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:32.667371 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffa147-0a6d-4dda-b23b-ef15093607c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:32.668048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ffa147-0a6d-4dda-b23b-ef15093607c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:32.670059 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ffa147-0a6d-4dda-b23b-ef15093607c8 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:32.767950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1303/2634] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:53:32 2022] POST /identity/v3/users => generated 287 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:32.769099 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7787d-d3ca-4468-9170-1aad95426858 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:32.779475 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7787d-d3ca-4468-9170-1aad95426858 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:32.784375 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b72e5-3573-49fc-a1f2-8431f15bd937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:32.785165 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b72e5-3573-49fc-a1f2-8431f15bd937 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:32.785525 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b72e5-3573-49fc-a1f2-8431f15bd937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:32.785814 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b72e5-3573-49fc-a1f2-8431f15bd937 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:32.786552 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b72e5-3573-49fc-a1f2-8431f15bd937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:32.789008 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b72e5-3573-49fc-a1f2-8431f15bd937 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:32.799596 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1332/2635] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:53:32.804506 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffa147-0a6d-4dda-b23b-ef15093607c8 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:32.814837 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffa147-0a6d-4dda-b23b-ef15093607c8 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:32.819959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6063a9-b7c4-44cb-bb31-c614ce241dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:32.820654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6063a9-b7c4-44cb-bb31-c614ce241dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:32.820946 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6063a9-b7c4-44cb-bb31-c614ce241dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:32.821224 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6063a9-b7c4-44cb-bb31-c614ce241dd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:32.821953 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6063a9-b7c4-44cb-bb31-c614ce241dd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:32.823677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6063a9-b7c4-44cb-bb31-c614ce241dd0 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:32.833734 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1304/2636] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:32.838142 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b72e5-3573-49fc-a1f2-8431f15bd937 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:32.848066 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b72e5-3573-49fc-a1f2-8431f15bd937 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:32.853309 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c9713-b30c-4f5e-9f8b-588b993b6ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:32.857348 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c9713-b30c-4f5e-9f8b-588b993b6ff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:32.857348 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c9713-b30c-4f5e-9f8b-588b993b6ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:32.857348 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c9713-b30c-4f5e-9f8b-588b993b6ff1 admin admin] PATH_INFO: `/v3/projects/9e78604c4b4f48a29a7b549c9e56f57d/users/1962367b0826422d86e08c72b4efe169/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:32.870065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c9713-b30c-4f5e-9f8b-588b993b6ff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e78604c4b4f48a29a7b549c9e56f57d, user_id=1962367b0826422d86e08c72b4efe169, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:32.872746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c9713-b30c-4f5e-9f8b-588b993b6ff1 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:32.940066 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1333/2637] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:53:32 2022] PUT /identity/v3/projects/9e78604c4b4f48a29a7b549c9e56f57d/users/1962367b0826422d86e08c72b4efe169/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:32.945286 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e6983-e014-401c-ad51-47c0f4e080d0 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:32.945579 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e6983-e014-401c-ad51-47c0f4e080d0 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:32.945860 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e6983-e014-401c-ad51-47c0f4e080d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:32.963397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-0a3e6983-e014-401c-ad51-47c0f4e080d0 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:32.964605 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-0a3e6983-e014-401c-ad51-47c0f4e080d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:33.023508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1305/2638] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:33.033338 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c9713-b30c-4f5e-9f8b-588b993b6ff1 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:33.034998 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-c81c9713-b30c-4f5e-9f8b-588b993b6ff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:33.092630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c9713-b30c-4f5e-9f8b-588b993b6ff1 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:33.096510 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d8f68-7a2d-4b78-a055-ceee50b87c9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:33.097230 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d8f68-7a2d-4b78-a055-ceee50b87c9c service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:33.097523 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d8f68-7a2d-4b78-a055-ceee50b87c9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:33.097839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d8f68-7a2d-4b78-a055-ceee50b87c9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:33.108465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d8f68-7a2d-4b78-a055-ceee50b87c9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:33.119061 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d8f68-7a2d-4b78-a055-ceee50b87c9c service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:33.119724 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1334/2639] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:53:34.315536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db89ee-46dc-41ff-bbf5-15aa6d5e53c4 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:34.316089 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db89ee-46dc-41ff-bbf5-15aa6d5e53c4 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:34.316089 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db89ee-46dc-41ff-bbf5-15aa6d5e53c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:34.344972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-97db89ee-46dc-41ff-bbf5-15aa6d5e53c4 None None] MFA Rules not processed for user `1962367b0826422d86e08c72b4efe169`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:34.346212 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-97db89ee-46dc-41ff-bbf5-15aa6d5e53c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:34.359046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:34.414682 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1306/2640] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:34.419148 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d8f68-7a2d-4b78-a055-ceee50b87c9c service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:34.430242 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d8f68-7a2d-4b78-a055-ceee50b87c9c service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:34.434966 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d995a7-a15d-4d28-a6e1-9387a6e6d39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:34.435702 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d995a7-a15d-4d28-a6e1-9387a6e6d39d admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:34.435991 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d995a7-a15d-4d28-a6e1-9387a6e6d39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:34.436304 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d995a7-a15d-4d28-a6e1-9387a6e6d39d admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:34.437043 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d995a7-a15d-4d28-a6e1-9387a6e6d39d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:34.439018 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d995a7-a15d-4d28-a6e1-9387a6e6d39d admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:34.524701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1335/2641] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:53:34 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:34.528362 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db89ee-46dc-41ff-bbf5-15aa6d5e53c4 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:34.539518 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db89ee-46dc-41ff-bbf5-15aa6d5e53c4 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:34.595554 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aea148-a910-4202-bc1f-15c39c1c2dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:34.596152 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aea148-a910-4202-bc1f-15c39c1c2dc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:34.596441 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aea148-a910-4202-bc1f-15c39c1c2dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:34.596745 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aea148-a910-4202-bc1f-15c39c1c2dc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:34.597461 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aea148-a910-4202-bc1f-15c39c1c2dc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:34.599404 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aea148-a910-4202-bc1f-15c39c1c2dc5 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:34.691460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1307/2642] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:53:34 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:34.696197 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d995a7-a15d-4d28-a6e1-9387a6e6d39d admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:34.706428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d995a7-a15d-4d28-a6e1-9387a6e6d39d admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:34.712202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4887a-949f-4242-a81a-afe955d3382f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:34.712929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4887a-949f-4242-a81a-afe955d3382f admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:34.713220 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4887a-949f-4242-a81a-afe955d3382f admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:34.713502 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4887a-949f-4242-a81a-afe955d3382f admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:34.714239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4887a-949f-4242-a81a-afe955d3382f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:34.716446 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4887a-949f-4242-a81a-afe955d3382f admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:34.729677 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1336/2643] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:53:34.733933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aea148-a910-4202-bc1f-15c39c1c2dc5 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:34.744792 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aea148-a910-4202-bc1f-15c39c1c2dc5 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:34.749467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de69dba-0bcc-477d-958c-87b18fc9647c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:34.751164 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de69dba-0bcc-477d-958c-87b18fc9647c admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:34.751164 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de69dba-0bcc-477d-958c-87b18fc9647c admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:34.751164 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de69dba-0bcc-477d-958c-87b18fc9647c admin admin] PATH_INFO: `/v3/projects/66ee6ab91a8146bc8980d510b5ad657c/users/4bd1464a3893472eac561ab91882f0c3/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:34.766079 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de69dba-0bcc-477d-958c-87b18fc9647c admin admin] RBAC: Authorizing `identity:create_grant(project_id=66ee6ab91a8146bc8980d510b5ad657c, user_id=4bd1464a3893472eac561ab91882f0c3, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:34.768804 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de69dba-0bcc-477d-958c-87b18fc9647c admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:34.863096 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1308/2644] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:53:34 2022] PUT /identity/v3/projects/66ee6ab91a8146bc8980d510b5ad657c/users/4bd1464a3893472eac561ab91882f0c3/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:34.867672 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4887a-949f-4242-a81a-afe955d3382f admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:34.877841 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4887a-949f-4242-a81a-afe955d3382f admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:34.922666 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:34.935163 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3f0a7-bdda-4ea8-90db-5fff23f79a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:34.935857 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3f0a7-bdda-4ea8-90db-5fff23f79a4d admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:34.936142 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3f0a7-bdda-4ea8-90db-5fff23f79a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:34.936422 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3f0a7-bdda-4ea8-90db-5fff23f79a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:34.937182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3f0a7-bdda-4ea8-90db-5fff23f79a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:34.938779 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3f0a7-bdda-4ea8-90db-5fff23f79a4d admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:34.948927 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1337/2645] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:53:34.953199 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de69dba-0bcc-477d-958c-87b18fc9647c admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:34.963945 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de69dba-0bcc-477d-958c-87b18fc9647c admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:34.968829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5091593-8fd9-41f8-9b6c-d09b6cf31e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:34.969740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5091593-8fd9-41f8-9b6c-d09b6cf31e9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:34.974624 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5091593-8fd9-41f8-9b6c-d09b6cf31e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:34.974624 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5091593-8fd9-41f8-9b6c-d09b6cf31e9a admin admin] PATH_INFO: `/v3/projects/66ee6ab91a8146bc8980d510b5ad657c/users/4bd1464a3893472eac561ab91882f0c3/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:34.974624 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5091593-8fd9-41f8-9b6c-d09b6cf31e9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=66ee6ab91a8146bc8980d510b5ad657c, user_id=4bd1464a3893472eac561ab91882f0c3, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:34.975996 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5091593-8fd9-41f8-9b6c-d09b6cf31e9a admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:34.986580 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1309/2646] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:53:34 2022] PUT /identity/v3/projects/66ee6ab91a8146bc8980d510b5ad657c/users/4bd1464a3893472eac561ab91882f0c3/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:35.304496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3f0a7-bdda-4ea8-90db-5fff23f79a4d admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:35.304496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-60e3f0a7-bdda-4ea8-90db-5fff23f79a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:35.314326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3f0a7-bdda-4ea8-90db-5fff23f79a4d admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:35.376916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1bcd6c-08bd-4940-8037-178f8a0cfdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:35.377724 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1bcd6c-08bd-4940-8037-178f8a0cfdb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:35.378012 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1bcd6c-08bd-4940-8037-178f8a0cfdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:35.378291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1bcd6c-08bd-4940-8037-178f8a0cfdb1 admin admin] PATH_INFO: `/v3/users/8744ee74dd4143b39a87a04405b8dc9b` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:35.379507 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1bcd6c-08bd-4940-8037-178f8a0cfdb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8744ee74dd4143b39a87a04405b8dc9b)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:35.381531 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1bcd6c-08bd-4940-8037-178f8a0cfdb1 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:35.558207 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2b1bcd6c-08bd-4940-8037-178f8a0cfdb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8744ee74dd4143b39a87a04405b8dc9b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:35.571905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2b1bcd6c-08bd-4940-8037-178f8a0cfdb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8744ee74dd4143b39a87a04405b8dc9b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:35.572691 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2b1bcd6c-08bd-4940-8037-178f8a0cfdb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8744ee74dd4143b39a87a04405b8dc9b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:35.573433 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2b1bcd6c-08bd-4940-8037-178f8a0cfdb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8744ee74dd4143b39a87a04405b8dc9b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:35.581410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-2b1bcd6c-08bd-4940-8037-178f8a0cfdb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8744ee74dd4143b39a87a04405b8dc9b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:35.680006 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1338/2647] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:53:35 2022] DELETE /identity/v3/users/8744ee74dd4143b39a87a04405b8dc9b => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:35.684466 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9db50-f22a-4344-85ec-7d5a5e353843 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:35.684964 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9db50-f22a-4344-85ec-7d5a5e353843 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:35.685062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9db50-f22a-4344-85ec-7d5a5e353843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:35.703631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-3dc9db50-f22a-4344-85ec-7d5a5e353843 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:35.704925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3dc9db50-f22a-4344-85ec-7d5a5e353843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:35.765350 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1310/2648] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:35.776030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1bcd6c-08bd-4940-8037-178f8a0cfdb1 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:35.777490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-2b1bcd6c-08bd-4940-8037-178f8a0cfdb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:35.837258 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1bcd6c-08bd-4940-8037-178f8a0cfdb1 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:35.840908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ddd38-2626-4923-a9e1-13cb024169b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:35.841565 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ddd38-2626-4923-a9e1-13cb024169b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:35.841855 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ddd38-2626-4923-a9e1-13cb024169b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:35.842138 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ddd38-2626-4923-a9e1-13cb024169b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:35.851698 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ddd38-2626-4923-a9e1-13cb024169b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:35.853255 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ddd38-2626-4923-a9e1-13cb024169b0 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:35.863303 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1339/2649] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:53:36.327207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9db50-f22a-4344-85ec-7d5a5e353843 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:36.329253 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3dc9db50-f22a-4344-85ec-7d5a5e353843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:36.347435 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9db50-f22a-4344-85ec-7d5a5e353843 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:36.353972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b5a08-1481-4e5b-ae6a-753a99cfd800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:36.354994 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b5a08-1481-4e5b-ae6a-753a99cfd800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:36.355378 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b5a08-1481-4e5b-ae6a-753a99cfd800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:36.355756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b5a08-1481-4e5b-ae6a-753a99cfd800 admin admin] PATH_INFO: `/v3/projects/112025b81f6f4005ad293939972f0576` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:36.357109 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b5a08-1481-4e5b-ae6a-753a99cfd800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=112025b81f6f4005ad293939972f0576)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:36.359233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b5a08-1481-4e5b-ae6a-753a99cfd800 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:36.362392 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:36.404363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31836c9-680a-4243-9cd0-e7ac1e1c4581 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:36.404681 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31836c9-680a-4243-9cd0-e7ac1e1c4581 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:36.404968 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31836c9-680a-4243-9cd0-e7ac1e1c4581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:36.421236 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:36.439595 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-a31836c9-680a-4243-9cd0-e7ac1e1c4581 None None] MFA Rules not processed for user `4bd1464a3893472eac561ab91882f0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:36.440956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a31836c9-680a-4243-9cd0-e7ac1e1c4581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:36.491765 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fa1b5a08-1481-4e5b-ae6a-753a99cfd800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '112025b81f6f4005ad293939972f0576', 'request_id': 'req-fa1b5a08-1481-4e5b-ae6a-753a99cfd800'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:36.520088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1340/2650] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:53:36.532067 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31836c9-680a-4243-9cd0-e7ac1e1c4581 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:36.533748 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a31836c9-680a-4243-9cd0-e7ac1e1c4581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:36.560196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fa1b5a08-1481-4e5b-ae6a-753a99cfd800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '112025b81f6f4005ad293939972f0576', 'request_id': 'req-fa1b5a08-1481-4e5b-ae6a-753a99cfd800'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:36.571263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fa1b5a08-1481-4e5b-ae6a-753a99cfd800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '112025b81f6f4005ad293939972f0576', 'request_id': 'req-fa1b5a08-1481-4e5b-ae6a-753a99cfd800'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:36.590990 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fa1b5a08-1481-4e5b-ae6a-753a99cfd800 admin admin] The token cache is being invalidate because project 112025b81f6f4005ad293939972f0576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:53:36.591526 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fa1b5a08-1481-4e5b-ae6a-753a99cfd800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:36.592372 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fa1b5a08-1481-4e5b-ae6a-753a99cfd800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:36.594431 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1311/2651] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:53:36 2022] DELETE /identity/v3/projects/112025b81f6f4005ad293939972f0576 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:36.598324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b5a08-1481-4e5b-ae6a-753a99cfd800 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:36.601693 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-fa1b5a08-1481-4e5b-ae6a-753a99cfd800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:36.602184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31836c9-680a-4243-9cd0-e7ac1e1c4581 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:36.658898 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc51189-5714-4de4-8807-7406e7cb527b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:36.659559 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc51189-5714-4de4-8807-7406e7cb527b service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:36.659912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc51189-5714-4de4-8807-7406e7cb527b service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:36.660196 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc51189-5714-4de4-8807-7406e7cb527b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:36.663468 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-2bc51189-5714-4de4-8807-7406e7cb527b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:36.690192 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b5a08-1481-4e5b-ae6a-753a99cfd800 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:36.696542 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705bc93-142b-4f31-880e-8e9b302da0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:36.697447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705bc93-142b-4f31-880e-8e9b302da0e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:36.697732 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705bc93-142b-4f31-880e-8e9b302da0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:36.698009 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705bc93-142b-4f31-880e-8e9b302da0e1 admin admin] PATH_INFO: `/v3/users/5de2ea4a29fe4cd38ad5509a35f8016b` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:36.699244 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705bc93-142b-4f31-880e-8e9b302da0e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5de2ea4a29fe4cd38ad5509a35f8016b)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:36.701403 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705bc93-142b-4f31-880e-8e9b302da0e1 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:36.729445 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc51189-5714-4de4-8807-7406e7cb527b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:36.730938 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc51189-5714-4de4-8807-7406e7cb527b service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:36.741148 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1341/2652] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:53:36.913604 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-2705bc93-142b-4f31-880e-8e9b302da0e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5de2ea4a29fe4cd38ad5509a35f8016b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:36.925396 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-2705bc93-142b-4f31-880e-8e9b302da0e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5de2ea4a29fe4cd38ad5509a35f8016b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:36.996750 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-2705bc93-142b-4f31-880e-8e9b302da0e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5de2ea4a29fe4cd38ad5509a35f8016b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:36.997324 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-2705bc93-142b-4f31-880e-8e9b302da0e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5de2ea4a29fe4cd38ad5509a35f8016b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:36.997975 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-2705bc93-142b-4f31-880e-8e9b302da0e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5de2ea4a29fe4cd38ad5509a35f8016b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:37.010771 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1312/2653] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:53:36 2022] DELETE /identity/v3/users/5de2ea4a29fe4cd38ad5509a35f8016b => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:37.499413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc51189-5714-4de4-8807-7406e7cb527b service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:37.510907 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc51189-5714-4de4-8807-7406e7cb527b service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:37.572245 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87a915-776b-4fc9-a284-0216de5f488d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:37.573179 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87a915-776b-4fc9-a284-0216de5f488d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:37.573477 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87a915-776b-4fc9-a284-0216de5f488d admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:37.573746 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87a915-776b-4fc9-a284-0216de5f488d admin admin] PATH_INFO: `/v3/projects/49d65cdfe62e44e796942a744b5117e1` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:37.574973 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87a915-776b-4fc9-a284-0216de5f488d admin admin] RBAC: Authorizing `identity:delete_project(project_id=49d65cdfe62e44e796942a744b5117e1)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:37.576990 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87a915-776b-4fc9-a284-0216de5f488d admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:37.700613 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-4c87a915-776b-4fc9-a284-0216de5f488d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49d65cdfe62e44e796942a744b5117e1', 'request_id': 'req-4c87a915-776b-4fc9-a284-0216de5f488d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:37.706874 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-4c87a915-776b-4fc9-a284-0216de5f488d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49d65cdfe62e44e796942a744b5117e1', 'request_id': 'req-4c87a915-776b-4fc9-a284-0216de5f488d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:37.712861 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-4c87a915-776b-4fc9-a284-0216de5f488d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49d65cdfe62e44e796942a744b5117e1', 'request_id': 'req-4c87a915-776b-4fc9-a284-0216de5f488d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:37.796785 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-4c87a915-776b-4fc9-a284-0216de5f488d admin admin] The token cache is being invalidate because project 49d65cdfe62e44e796942a744b5117e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:53:37.796986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-4c87a915-776b-4fc9-a284-0216de5f488d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:37.797601 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-4c87a915-776b-4fc9-a284-0216de5f488d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:37.799161 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1342/2654] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:53:37 2022] DELETE /identity/v3/projects/49d65cdfe62e44e796942a744b5117e1 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:53:37.803385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705bc93-142b-4f31-880e-8e9b302da0e1 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:37.804867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-2705bc93-142b-4f31-880e-8e9b302da0e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:37.818112 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705bc93-142b-4f31-880e-8e9b302da0e1 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:37.825876 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:37.877037 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f549fd-d9d3-45ea-98c8-07ae3ecc99d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:37.877910 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f549fd-d9d3-45ea-98c8-07ae3ecc99d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:37.878194 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f549fd-d9d3-45ea-98c8-07ae3ecc99d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:37.878467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f549fd-d9d3-45ea-98c8-07ae3ecc99d7 admin admin] PATH_INFO: `/v3/users/ae050adbc8ec4f34b093564dd7580ba0` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:37.879651 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f549fd-d9d3-45ea-98c8-07ae3ecc99d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae050adbc8ec4f34b093564dd7580ba0)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:37.882337 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f549fd-d9d3-45ea-98c8-07ae3ecc99d7 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:37.963173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:38.102164 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-18f549fd-d9d3-45ea-98c8-07ae3ecc99d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae050adbc8ec4f34b093564dd7580ba0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:38.116076 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-18f549fd-d9d3-45ea-98c8-07ae3ecc99d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae050adbc8ec4f34b093564dd7580ba0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:38.205616 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-18f549fd-d9d3-45ea-98c8-07ae3ecc99d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae050adbc8ec4f34b093564dd7580ba0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:38.206228 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-18f549fd-d9d3-45ea-98c8-07ae3ecc99d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae050adbc8ec4f34b093564dd7580ba0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:38.206822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-18f549fd-d9d3-45ea-98c8-07ae3ecc99d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae050adbc8ec4f34b093564dd7580ba0'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:38.220375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1313/2655] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:53:37 2022] DELETE /identity/v3/users/ae050adbc8ec4f34b093564dd7580ba0 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:38.465195 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87a915-776b-4fc9-a284-0216de5f488d admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:38.466953 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-4c87a915-776b-4fc9-a284-0216de5f488d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:38.526986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87a915-776b-4fc9-a284-0216de5f488d admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:38.571267 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:38.575579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69563fb6-72e1-4516-a8e9-103db7f55514 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:38.576233 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69563fb6-72e1-4516-a8e9-103db7f55514 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:38.576515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69563fb6-72e1-4516-a8e9-103db7f55514 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:38.576817 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69563fb6-72e1-4516-a8e9-103db7f55514 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:38.578635 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-69563fb6-72e1-4516-a8e9-103db7f55514 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:38.639547 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69563fb6-72e1-4516-a8e9-103db7f55514 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:38.644381 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69563fb6-72e1-4516-a8e9-103db7f55514 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:38.647638 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f549fd-d9d3-45ea-98c8-07ae3ecc99d7 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:38.649079 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-18f549fd-d9d3-45ea-98c8-07ae3ecc99d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:38.652916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1343/2656] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:53:38.716200 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f549fd-d9d3-45ea-98c8-07ae3ecc99d7 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:38.720957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe50d8-8baf-4f31-a35f-5fbe6ef0710a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:38.721823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe50d8-8baf-4f31-a35f-5fbe6ef0710a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:38.722110 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe50d8-8baf-4f31-a35f-5fbe6ef0710a admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:38.722389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe50d8-8baf-4f31-a35f-5fbe6ef0710a admin admin] PATH_INFO: `/v3/projects/13f99f7632f04ce3b29f4a69b8cccbe1` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:38.723570 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe50d8-8baf-4f31-a35f-5fbe6ef0710a admin admin] RBAC: Authorizing `identity:delete_project(project_id=13f99f7632f04ce3b29f4a69b8cccbe1)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:38.726240 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe50d8-8baf-4f31-a35f-5fbe6ef0710a admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:38.859094 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6cbe50d8-8baf-4f31-a35f-5fbe6ef0710a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13f99f7632f04ce3b29f4a69b8cccbe1', 'request_id': 'req-6cbe50d8-8baf-4f31-a35f-5fbe6ef0710a'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:38.928924 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6cbe50d8-8baf-4f31-a35f-5fbe6ef0710a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13f99f7632f04ce3b29f4a69b8cccbe1', 'request_id': 'req-6cbe50d8-8baf-4f31-a35f-5fbe6ef0710a'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:38.935702 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6cbe50d8-8baf-4f31-a35f-5fbe6ef0710a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13f99f7632f04ce3b29f4a69b8cccbe1', 'request_id': 'req-6cbe50d8-8baf-4f31-a35f-5fbe6ef0710a'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:38.946514 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6cbe50d8-8baf-4f31-a35f-5fbe6ef0710a admin admin] The token cache is being invalidate because project 13f99f7632f04ce3b29f4a69b8cccbe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:53:38.946810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6cbe50d8-8baf-4f31-a35f-5fbe6ef0710a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:38.947473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6cbe50d8-8baf-4f31-a35f-5fbe6ef0710a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:38.949195 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1314/2657] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:53:38 2022] DELETE /identity/v3/projects/13f99f7632f04ce3b29f4a69b8cccbe1 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:38.954163 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e4c2e-0b48-47ca-8f3e-da3e13ae321d None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:38.954626 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e4c2e-0b48-47ca-8f3e-da3e13ae321d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:38.954753 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e4c2e-0b48-47ca-8f3e-da3e13ae321d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:38.972668 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-db1e4c2e-0b48-47ca-8f3e-da3e13ae321d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:38.974402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-db1e4c2e-0b48-47ca-8f3e-da3e13ae321d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:39.034165 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1344/2658] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:39.042846 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb50f5a-4944-40df-84ec-26e1df9d30d6 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:39.043142 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb50f5a-4944-40df-84ec-26e1df9d30d6 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:39.043427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb50f5a-4944-40df-84ec-26e1df9d30d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:39.061782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-0bb50f5a-4944-40df-84ec-26e1df9d30d6 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:39.063002 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-0bb50f5a-4944-40df-84ec-26e1df9d30d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:39.071596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1315/2659] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:39.076088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e4c2e-0b48-47ca-8f3e-da3e13ae321d None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:39.087812 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e4c2e-0b48-47ca-8f3e-da3e13ae321d None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:39.092777 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439bc99-cb64-479c-8186-26b23fb9852a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:39.093454 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439bc99-cb64-479c-8186-26b23fb9852a admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:39.093744 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439bc99-cb64-479c-8186-26b23fb9852a admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:39.094037 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439bc99-cb64-479c-8186-26b23fb9852a admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:39.094762 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439bc99-cb64-479c-8186-26b23fb9852a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:53:39.095071 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439bc99-cb64-479c-8186-26b23fb9852a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:39.096784 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439bc99-cb64-479c-8186-26b23fb9852a admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:39.110054 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1345/2660] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:53:39.114221 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb50f5a-4944-40df-84ec-26e1df9d30d6 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:39.132991 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb50f5a-4944-40df-84ec-26e1df9d30d6 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:39.136084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ebacc-cfa1-481d-b648-7e6ea020247d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:39.136783 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ebacc-cfa1-481d-b648-7e6ea020247d admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:39.137074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ebacc-cfa1-481d-b648-7e6ea020247d admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:39.137352 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ebacc-cfa1-481d-b648-7e6ea020247d admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:39.138203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ebacc-cfa1-481d-b648-7e6ea020247d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:39.140014 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ebacc-cfa1-481d-b648-7e6ea020247d admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:39.232004 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1316/2661] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:53:39 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:39.233194 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439bc99-cb64-479c-8186-26b23fb9852a admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:39.243838 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439bc99-cb64-479c-8186-26b23fb9852a admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:39.304003 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ccbf1c-7787-4642-baa7-d5e24d4f07cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:39.304735 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccbf1c-7787-4642-baa7-d5e24d4f07cc admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:39.305067 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccbf1c-7787-4642-baa7-d5e24d4f07cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:39.305353 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccbf1c-7787-4642-baa7-d5e24d4f07cc admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:39.306065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ccbf1c-7787-4642-baa7-d5e24d4f07cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:39.307996 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ccbf1c-7787-4642-baa7-d5e24d4f07cc admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:39.461070 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1346/2662] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:53:39 2022] POST /identity/v3/users => generated 287 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:53:39.465101 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ebacc-cfa1-481d-b648-7e6ea020247d admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:39.475810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ebacc-cfa1-481d-b648-7e6ea020247d admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:39.480648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3ba51-b907-4a6d-bb62-cf7487a031b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:39.481309 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3ba51-b907-4a6d-bb62-cf7487a031b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:39.481614 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3ba51-b907-4a6d-bb62-cf7487a031b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:39.481927 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3ba51-b907-4a6d-bb62-cf7487a031b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:39.482659 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3ba51-b907-4a6d-bb62-cf7487a031b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:39.484256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3ba51-b907-4a6d-bb62-cf7487a031b6 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:39.494880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1317/2663] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:39.498814 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ccbf1c-7787-4642-baa7-d5e24d4f07cc admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:39.509493 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ccbf1c-7787-4642-baa7-d5e24d4f07cc admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:39.514346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5683422-56ba-4262-ae8d-70abd9c41d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:39.515015 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5683422-56ba-4262-ae8d-70abd9c41d6a admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:39.515306 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5683422-56ba-4262-ae8d-70abd9c41d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:39.515590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5683422-56ba-4262-ae8d-70abd9c41d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:39.516353 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5683422-56ba-4262-ae8d-70abd9c41d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:39.517998 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5683422-56ba-4262-ae8d-70abd9c41d6a admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:39.528593 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1347/2664] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:39.532788 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3ba51-b907-4a6d-bb62-cf7487a031b6 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:39.543151 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3ba51-b907-4a6d-bb62-cf7487a031b6 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:39.547830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c4425-0a4f-420e-b02f-d309b3ab96ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:39.548770 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c4425-0a4f-420e-b02f-d309b3ab96ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:39.549101 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c4425-0a4f-420e-b02f-d309b3ab96ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:39.549356 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c4425-0a4f-420e-b02f-d309b3ab96ad admin admin] PATH_INFO: `/v3/projects/7cee5e6784454293bc02e0e595d58b22/users/2dcf17b8ad9349458e3328ebc2984902/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:39.563779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c4425-0a4f-420e-b02f-d309b3ab96ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cee5e6784454293bc02e0e595d58b22, user_id=2dcf17b8ad9349458e3328ebc2984902, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:39.566495 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c4425-0a4f-420e-b02f-d309b3ab96ad admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:39.591528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:39.657559 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1318/2665] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:53:39 2022] PUT /identity/v3/projects/7cee5e6784454293bc02e0e595d58b22/users/2dcf17b8ad9349458e3328ebc2984902/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:39.662961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29bf36-e66a-4bf8-8144-d98ed842ae83 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:39.663263 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29bf36-e66a-4bf8-8144-d98ed842ae83 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:39.663573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29bf36-e66a-4bf8-8144-d98ed842ae83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:39.681479 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-ef29bf36-e66a-4bf8-8144-d98ed842ae83 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:39.682678 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ef29bf36-e66a-4bf8-8144-d98ed842ae83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:39.743285 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1348/2666] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:39.753046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c4425-0a4f-420e-b02f-d309b3ab96ad admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:39.754720 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-796c4425-0a4f-420e-b02f-d309b3ab96ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:39.810654 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c4425-0a4f-420e-b02f-d309b3ab96ad admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:39.814389 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5615dd4-3fd0-4778-8dec-a874a5b4b4bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:39.815023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5615dd4-3fd0-4778-8dec-a874a5b4b4bc service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:39.815305 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5615dd4-3fd0-4778-8dec-a874a5b4b4bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:39.815579 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5615dd4-3fd0-4778-8dec-a874a5b4b4bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:39.825220 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5615dd4-3fd0-4778-8dec-a874a5b4b4bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:39.826641 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5615dd4-3fd0-4778-8dec-a874a5b4b4bc service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:39.836045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1319/2667] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:53:40.218086 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:41.151831 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac597e-25e2-4bf4-96f3-a6b48c1153d5 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:41.152123 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac597e-25e2-4bf4-96f3-a6b48c1153d5 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:41.152400 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac597e-25e2-4bf4-96f3-a6b48c1153d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:41.180220 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-dbac597e-25e2-4bf4-96f3-a6b48c1153d5 None None] MFA Rules not processed for user `2dcf17b8ad9349458e3328ebc2984902`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:41.181579 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-dbac597e-25e2-4bf4-96f3-a6b48c1153d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:41.248417 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1349/2668] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:41.259764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5615dd4-3fd0-4778-8dec-a874a5b4b4bc service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:41.261196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e5615dd4-3fd0-4778-8dec-a874a5b4b4bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:41.320237 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5615dd4-3fd0-4778-8dec-a874a5b4b4bc service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:41.324012 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2457f2-8740-48ee-a4cc-85ecddf8058d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:41.324680 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2457f2-8740-48ee-a4cc-85ecddf8058d service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:41.324965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2457f2-8740-48ee-a4cc-85ecddf8058d service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:41.325247 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2457f2-8740-48ee-a4cc-85ecddf8058d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:41.334617 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2457f2-8740-48ee-a4cc-85ecddf8058d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:41.336043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2457f2-8740-48ee-a4cc-85ecddf8058d service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:41.345027 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1320/2669] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:53:43.115272 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290be4c2-0ed2-4194-9594-7bcd78e86423 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:43.115799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290be4c2-0ed2-4194-9594-7bcd78e86423 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:43.115963 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290be4c2-0ed2-4194-9594-7bcd78e86423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:43.134226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-290be4c2-0ed2-4194-9594-7bcd78e86423 None None] MFA Rules not processed for user `2dcf17b8ad9349458e3328ebc2984902`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:43.135413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-290be4c2-0ed2-4194-9594-7bcd78e86423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:43.142303 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1350/2670] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:51.340834 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:53.128858 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=60 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:54.560473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2457f2-8740-48ee-a4cc-85ecddf8058d service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:54.562082 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-8b2457f2-8740-48ee-a4cc-85ecddf8058d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:54.578053 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2457f2-8740-48ee-a4cc-85ecddf8058d service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:54.583161 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4a212-0d31-4227-a414-c84b63680a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:54.584138 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4a212-0d31-4227-a414-c84b63680a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:54.584493 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4a212-0d31-4227-a414-c84b63680a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:54.584893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4a212-0d31-4227-a414-c84b63680a5f admin admin] PATH_INFO: `/v3/users/1962367b0826422d86e08c72b4efe169` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:54.586163 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf4a212-0d31-4227-a414-c84b63680a5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1962367b0826422d86e08c72b4efe169)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:54.589045 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf4a212-0d31-4227-a414-c84b63680a5f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:54.753887 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6cf4a212-0d31-4227-a414-c84b63680a5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1962367b0826422d86e08c72b4efe169'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:54.767320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6cf4a212-0d31-4227-a414-c84b63680a5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1962367b0826422d86e08c72b4efe169'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:54.836653 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6cf4a212-0d31-4227-a414-c84b63680a5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1962367b0826422d86e08c72b4efe169'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:54.837275 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6cf4a212-0d31-4227-a414-c84b63680a5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1962367b0826422d86e08c72b4efe169'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:54.837998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6cf4a212-0d31-4227-a414-c84b63680a5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1962367b0826422d86e08c72b4efe169'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:54.852637 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1321/2671] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:53:54 2022] DELETE /identity/v3/users/1962367b0826422d86e08c72b4efe169 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:55.146749 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290be4c2-0ed2-4194-9594-7bcd78e86423 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:55.172496 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290be4c2-0ed2-4194-9594-7bcd78e86423 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:55.238164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b4553-8f2f-4a49-bf2c-6f1bec5e7b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:55.239038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b4553-8f2f-4a49-bf2c-6f1bec5e7b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:55.240511 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b4553-8f2f-4a49-bf2c-6f1bec5e7b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:55.240511 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b4553-8f2f-4a49-bf2c-6f1bec5e7b97 admin admin] PATH_INFO: `/v3/projects/9e78604c4b4f48a29a7b549c9e56f57d` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:55.240954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b4553-8f2f-4a49-bf2c-6f1bec5e7b97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e78604c4b4f48a29a7b549c9e56f57d)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:55.242864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b4553-8f2f-4a49-bf2c-6f1bec5e7b97 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:55.366172 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-769b4553-8f2f-4a49-bf2c-6f1bec5e7b97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e78604c4b4f48a29a7b549c9e56f57d', 'request_id': 'req-769b4553-8f2f-4a49-bf2c-6f1bec5e7b97'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:55.373093 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-769b4553-8f2f-4a49-bf2c-6f1bec5e7b97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e78604c4b4f48a29a7b549c9e56f57d', 'request_id': 'req-769b4553-8f2f-4a49-bf2c-6f1bec5e7b97'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:55.378984 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-769b4553-8f2f-4a49-bf2c-6f1bec5e7b97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e78604c4b4f48a29a7b549c9e56f57d', 'request_id': 'req-769b4553-8f2f-4a49-bf2c-6f1bec5e7b97'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:55.444141 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-769b4553-8f2f-4a49-bf2c-6f1bec5e7b97 admin admin] The token cache is being invalidate because project 9e78604c4b4f48a29a7b549c9e56f57d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:53:55.444457 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-769b4553-8f2f-4a49-bf2c-6f1bec5e7b97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:55.445103 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-769b4553-8f2f-4a49-bf2c-6f1bec5e7b97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:55.446688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1351/2672] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:53:55 2022] DELETE /identity/v3/projects/9e78604c4b4f48a29a7b549c9e56f57d => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:55.456347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4a212-0d31-4227-a414-c84b63680a5f admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:55.457782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-6cf4a212-0d31-4227-a414-c84b63680a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:55.471601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4a212-0d31-4227-a414-c84b63680a5f admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:55.527066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99d593-5b6d-40b8-981a-23e5b3575101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:55.527965 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99d593-5b6d-40b8-981a-23e5b3575101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:55.528250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99d593-5b6d-40b8-981a-23e5b3575101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:55.528529 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99d593-5b6d-40b8-981a-23e5b3575101 admin admin] PATH_INFO: `/v3/users/4bd1464a3893472eac561ab91882f0c3` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:55.532796 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f99d593-5b6d-40b8-981a-23e5b3575101 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bd1464a3893472eac561ab91882f0c3)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:55.532796 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f99d593-5b6d-40b8-981a-23e5b3575101 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:55.718953 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3f99d593-5b6d-40b8-981a-23e5b3575101 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bd1464a3893472eac561ab91882f0c3'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:55.731457 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3f99d593-5b6d-40b8-981a-23e5b3575101 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bd1464a3893472eac561ab91882f0c3'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:55.796132 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3f99d593-5b6d-40b8-981a-23e5b3575101 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bd1464a3893472eac561ab91882f0c3'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:55.796773 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3f99d593-5b6d-40b8-981a-23e5b3575101 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bd1464a3893472eac561ab91882f0c3'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:55.797378 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3f99d593-5b6d-40b8-981a-23e5b3575101 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bd1464a3893472eac561ab91882f0c3'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:55.799514 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:55.810167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1322/2673] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:53:55 2022] DELETE /identity/v3/users/4bd1464a3893472eac561ab91882f0c3 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:56.059194 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b4553-8f2f-4a49-bf2c-6f1bec5e7b97 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:56.059194 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-769b4553-8f2f-4a49-bf2c-6f1bec5e7b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:56.071670 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b4553-8f2f-4a49-bf2c-6f1bec5e7b97 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:56.130647 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3ab5d-0726-4112-8915-42f4b51c24d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:56.131510 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3ab5d-0726-4112-8915-42f4b51c24d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:56.131797 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3ab5d-0726-4112-8915-42f4b51c24d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:56.132076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3ab5d-0726-4112-8915-42f4b51c24d4 admin admin] PATH_INFO: `/v3/projects/66ee6ab91a8146bc8980d510b5ad657c` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:56.133291 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b3ab5d-0726-4112-8915-42f4b51c24d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66ee6ab91a8146bc8980d510b5ad657c)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:56.135842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b3ab5d-0726-4112-8915-42f4b51c24d4 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:56.270826 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:56.282567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-88b3ab5d-0726-4112-8915-42f4b51c24d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66ee6ab91a8146bc8980d510b5ad657c', 'request_id': 'req-88b3ab5d-0726-4112-8915-42f4b51c24d4'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:56.288929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-88b3ab5d-0726-4112-8915-42f4b51c24d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66ee6ab91a8146bc8980d510b5ad657c', 'request_id': 'req-88b3ab5d-0726-4112-8915-42f4b51c24d4'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:56.294630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-88b3ab5d-0726-4112-8915-42f4b51c24d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66ee6ab91a8146bc8980d510b5ad657c', 'request_id': 'req-88b3ab5d-0726-4112-8915-42f4b51c24d4'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:56.390934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-88b3ab5d-0726-4112-8915-42f4b51c24d4 admin admin] The token cache is being invalidate because project 66ee6ab91a8146bc8980d510b5ad657c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:53:56.391135 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-88b3ab5d-0726-4112-8915-42f4b51c24d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:56.391776 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-88b3ab5d-0726-4112-8915-42f4b51c24d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:56.393336 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1352/2674] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:53:56 2022] DELETE /identity/v3/projects/66ee6ab91a8146bc8980d510b5ad657c => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:56.398360 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06b9fb-af1b-4774-8b99-569faa45888a None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:56.398642 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06b9fb-af1b-4774-8b99-569faa45888a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:56.398927 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06b9fb-af1b-4774-8b99-569faa45888a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:56.418558 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-cf06b9fb-af1b-4774-8b99-569faa45888a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:56.419758 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-cf06b9fb-af1b-4774-8b99-569faa45888a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:56.479611 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1323/2675] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:56.487437 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7bad1-f71d-419f-8303-35ab167e900a None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:56.487730 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7bad1-f71d-419f-8303-35ab167e900a None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:56.488220 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7bad1-f71d-419f-8303-35ab167e900a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:56.507347 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-5bd7bad1-f71d-419f-8303-35ab167e900a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:56.509535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-5bd7bad1-f71d-419f-8303-35ab167e900a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:56.516750 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1353/2676] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:56.520862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06b9fb-af1b-4774-8b99-569faa45888a None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:56.530366 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06b9fb-af1b-4774-8b99-569faa45888a None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:56.534845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd539499-420b-49d2-a62f-30a87232477f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:56.535488 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd539499-420b-49d2-a62f-30a87232477f admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:56.535770 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd539499-420b-49d2-a62f-30a87232477f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:56.536046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd539499-420b-49d2-a62f-30a87232477f admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:56.536802 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd539499-420b-49d2-a62f-30a87232477f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:53:56.537108 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd539499-420b-49d2-a62f-30a87232477f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:56.538713 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd539499-420b-49d2-a62f-30a87232477f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:56.550556 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1324/2677] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:53:56.554696 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7bad1-f71d-419f-8303-35ab167e900a None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:56.564189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7bad1-f71d-419f-8303-35ab167e900a None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:56.568826 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2008756-9068-49b1-ac34-10d3168788a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:56.569462 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2008756-9068-49b1-ac34-10d3168788a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:56.569745 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2008756-9068-49b1-ac34-10d3168788a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:56.570017 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2008756-9068-49b1-ac34-10d3168788a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:56.570689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2008756-9068-49b1-ac34-10d3168788a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:56.572777 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2008756-9068-49b1-ac34-10d3168788a8 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:56.648880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1354/2678] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:53:56 2022] POST /identity/v3/projects => generated 371 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:56.652708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd539499-420b-49d2-a62f-30a87232477f admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:56.662712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd539499-420b-49d2-a62f-30a87232477f admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:56.719434 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6ef5b-34d9-467f-9948-496010986da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:56.720103 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6ef5b-34d9-467f-9948-496010986da9 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:56.720386 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6ef5b-34d9-467f-9948-496010986da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:56.720692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6ef5b-34d9-467f-9948-496010986da9 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:56.721528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6ef5b-34d9-467f-9948-496010986da9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:56.723438 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6ef5b-34d9-467f-9948-496010986da9 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:56.831445 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1325/2679] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:53:56 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:56.835649 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2008756-9068-49b1-ac34-10d3168788a8 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:56.846334 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2008756-9068-49b1-ac34-10d3168788a8 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:56.851475 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5366ad-c367-45f8-85c4-65f301535489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:56.856853 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5366ad-c367-45f8-85c4-65f301535489 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:56.856853 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5366ad-c367-45f8-85c4-65f301535489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:56.856853 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5366ad-c367-45f8-85c4-65f301535489 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:56.856853 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5366ad-c367-45f8-85c4-65f301535489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:56.856853 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5366ad-c367-45f8-85c4-65f301535489 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:56.867254 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1355/2680] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:56.871721 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6ef5b-34d9-467f-9948-496010986da9 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:56.881736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6ef5b-34d9-467f-9948-496010986da9 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:56.886371 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be3637-28ee-4e01-9b1d-6053494342e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:56.887020 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be3637-28ee-4e01-9b1d-6053494342e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:56.887303 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be3637-28ee-4e01-9b1d-6053494342e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:56.887581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be3637-28ee-4e01-9b1d-6053494342e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:56.888303 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be3637-28ee-4e01-9b1d-6053494342e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:56.889940 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be3637-28ee-4e01-9b1d-6053494342e2 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:56.900347 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1326/2681] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:56.905078 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5366ad-c367-45f8-85c4-65f301535489 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:56.915223 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5366ad-c367-45f8-85c4-65f301535489 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:56.920490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd4f84-32aa-47c2-8e12-de02ebeb47d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:56.921499 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd4f84-32aa-47c2-8e12-de02ebeb47d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:56.921784 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd4f84-32aa-47c2-8e12-de02ebeb47d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:56.922058 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd4f84-32aa-47c2-8e12-de02ebeb47d2 admin admin] PATH_INFO: `/v3/projects/4d15e1b790674be98ef2c376040bf39f/users/079f872fbd934e51b69954b7f64645c7/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:56.937677 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd4f84-32aa-47c2-8e12-de02ebeb47d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d15e1b790674be98ef2c376040bf39f, user_id=079f872fbd934e51b69954b7f64645c7, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:56.942954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd4f84-32aa-47c2-8e12-de02ebeb47d2 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:57.025890 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1356/2682] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:53:56 2022] PUT /identity/v3/projects/4d15e1b790674be98ef2c376040bf39f/users/079f872fbd934e51b69954b7f64645c7/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:57.036373 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa84cc-6398-4a68-a7e5-e542ad4c4fd3 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:57.036746 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa84cc-6398-4a68-a7e5-e542ad4c4fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:57.037051 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa84cc-6398-4a68-a7e5-e542ad4c4fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:57.066097 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-4efa84cc-6398-4a68-a7e5-e542ad4c4fd3 None None] MFA Rules not processed for user `079f872fbd934e51b69954b7f64645c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:57.067321 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-4efa84cc-6398-4a68-a7e5-e542ad4c4fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:57.128678 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:57.148525 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1327/2683] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:57.160640 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd4f84-32aa-47c2-8e12-de02ebeb47d2 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:57.162376 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-14cd4f84-32aa-47c2-8e12-de02ebeb47d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:57.222491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd4f84-32aa-47c2-8e12-de02ebeb47d2 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:57.226255 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2623f-356e-49d0-aeb1-0fae0cef4c9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:57.226905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2623f-356e-49d0-aeb1-0fae0cef4c9d service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:57.227195 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2623f-356e-49d0-aeb1-0fae0cef4c9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:57.227476 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2623f-356e-49d0-aeb1-0fae0cef4c9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:57.237480 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb2623f-356e-49d0-aeb1-0fae0cef4c9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:57.238905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb2623f-356e-49d0-aeb1-0fae0cef4c9d service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:57.248065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1357/2684] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:53:57.269497 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa84cc-6398-4a68-a7e5-e542ad4c4fd3 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:57.332392 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa84cc-6398-4a68-a7e5-e542ad4c4fd3 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:57.336993 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ebd60-c8c7-416a-9978-3264d0defa0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:57.338486 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ebd60-c8c7-416a-9978-3264d0defa0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:57.338769 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ebd60-c8c7-416a-9978-3264d0defa0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:57.339039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ebd60-c8c7-416a-9978-3264d0defa0c admin admin] PATH_INFO: `/v3/users/079f872fbd934e51b69954b7f64645c7` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:57.340239 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ebd60-c8c7-416a-9978-3264d0defa0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=079f872fbd934e51b69954b7f64645c7)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:57.342204 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ebd60-c8c7-416a-9978-3264d0defa0c admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:57.536550 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-981ebd60-c8c7-416a-9978-3264d0defa0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '079f872fbd934e51b69954b7f64645c7'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:57.548524 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-981ebd60-c8c7-416a-9978-3264d0defa0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '079f872fbd934e51b69954b7f64645c7'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:57.647428 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-981ebd60-c8c7-416a-9978-3264d0defa0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '079f872fbd934e51b69954b7f64645c7'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:57.648116 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-981ebd60-c8c7-416a-9978-3264d0defa0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '079f872fbd934e51b69954b7f64645c7'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:57.648776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-981ebd60-c8c7-416a-9978-3264d0defa0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '079f872fbd934e51b69954b7f64645c7'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:57.661542 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1328/2685] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:53:57 2022] DELETE /identity/v3/users/079f872fbd934e51b69954b7f64645c7 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:57.666243 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f6e38-4eea-4b03-9ac6-278553303924 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:57.666708 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f6e38-4eea-4b03-9ac6-278553303924 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:57.666824 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f6e38-4eea-4b03-9ac6-278553303924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:57.683794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-0d1f6e38-4eea-4b03-9ac6-278553303924 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:57.684946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-0d1f6e38-4eea-4b03-9ac6-278553303924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:57.737825 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:57.746814 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1358/2686] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:57.756397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ebd60-c8c7-416a-9978-3264d0defa0c admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:57.757873 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-981ebd60-c8c7-416a-9978-3264d0defa0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:57.817184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ebd60-c8c7-416a-9978-3264d0defa0c admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:57.820878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05cb19-98d4-4d2e-8d7d-7dc1bd2c3d34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:57.821522 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05cb19-98d4-4d2e-8d7d-7dc1bd2c3d34 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:57.822398 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05cb19-98d4-4d2e-8d7d-7dc1bd2c3d34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:57.822676 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05cb19-98d4-4d2e-8d7d-7dc1bd2c3d34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:57.831938 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05cb19-98d4-4d2e-8d7d-7dc1bd2c3d34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:57.833405 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05cb19-98d4-4d2e-8d7d-7dc1bd2c3d34 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:57.843443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1329/2687] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:53:58.257914 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f6e38-4eea-4b03-9ac6-278553303924 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:58.259801 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-0d1f6e38-4eea-4b03-9ac6-278553303924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:58.272547 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f6e38-4eea-4b03-9ac6-278553303924 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:58.277983 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a06a9e-c7b2-474a-aecd-e9a120293a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:58.284909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a06a9e-c7b2-474a-aecd-e9a120293a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:58.284909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a06a9e-c7b2-474a-aecd-e9a120293a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:58.284909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a06a9e-c7b2-474a-aecd-e9a120293a71 admin admin] PATH_INFO: `/v3/projects/4d15e1b790674be98ef2c376040bf39f` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:58.284909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a06a9e-c7b2-474a-aecd-e9a120293a71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d15e1b790674be98ef2c376040bf39f)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:58.284909 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a06a9e-c7b2-474a-aecd-e9a120293a71 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:58.431712 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d8a06a9e-c7b2-474a-aecd-e9a120293a71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d15e1b790674be98ef2c376040bf39f', 'request_id': 'req-d8a06a9e-c7b2-474a-aecd-e9a120293a71'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:58.437941 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d8a06a9e-c7b2-474a-aecd-e9a120293a71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d15e1b790674be98ef2c376040bf39f', 'request_id': 'req-d8a06a9e-c7b2-474a-aecd-e9a120293a71'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:58.443493 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d8a06a9e-c7b2-474a-aecd-e9a120293a71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d15e1b790674be98ef2c376040bf39f', 'request_id': 'req-d8a06a9e-c7b2-474a-aecd-e9a120293a71'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:58.518667 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d8a06a9e-c7b2-474a-aecd-e9a120293a71 admin admin] The token cache is being invalidate because project 4d15e1b790674be98ef2c376040bf39f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:53:58.518869 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d8a06a9e-c7b2-474a-aecd-e9a120293a71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:58.519519 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d8a06a9e-c7b2-474a-aecd-e9a120293a71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:53:58.521141 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1359/2688] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:53:58 2022] DELETE /identity/v3/projects/4d15e1b790674be98ef2c376040bf39f => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:58.526635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68f58c-9163-4d62-b648-9bc76da3833f None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:58.526929 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68f58c-9163-4d62-b648-9bc76da3833f None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:58.527211 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68f58c-9163-4d62-b648-9bc76da3833f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:58.545898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-2d68f58c-9163-4d62-b648-9bc76da3833f None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:58.547106 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-2d68f58c-9163-4d62-b648-9bc76da3833f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:58.551373 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:58.606780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1330/2689] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:58.615359 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daa389-4764-474a-800f-f15d3cb1b4c2 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:58.615655 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daa389-4764-474a-800f-f15d3cb1b4c2 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:58.615956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daa389-4764-474a-800f-f15d3cb1b4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:58.634354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-59daa389-4764-474a-800f-f15d3cb1b4c2 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:58.635679 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-59daa389-4764-474a-800f-f15d3cb1b4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:58.643602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1360/2690] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:58.647824 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68f58c-9163-4d62-b648-9bc76da3833f None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:58.658162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68f58c-9163-4d62-b648-9bc76da3833f None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:58.663439 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a3152-f86b-4fb1-b77f-1c97d99e2531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:58.664084 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a3152-f86b-4fb1-b77f-1c97d99e2531 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:58.664368 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a3152-f86b-4fb1-b77f-1c97d99e2531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:58.664677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a3152-f86b-4fb1-b77f-1c97d99e2531 admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:58.665401 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774a3152-f86b-4fb1-b77f-1c97d99e2531 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:53:58.665711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774a3152-f86b-4fb1-b77f-1c97d99e2531 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:58.667926 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774a3152-f86b-4fb1-b77f-1c97d99e2531 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:58.679289 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1331/2691] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:53:58.683588 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59daa389-4764-474a-800f-f15d3cb1b4c2 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:58.692899 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59daa389-4764-474a-800f-f15d3cb1b4c2 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:58.697569 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d99c3-d8eb-4cbf-8838-ae7d010396ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:58.698723 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d99c3-d8eb-4cbf-8838-ae7d010396ac admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:58.698989 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d99c3-d8eb-4cbf-8838-ae7d010396ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:58.699263 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d99c3-d8eb-4cbf-8838-ae7d010396ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:58.699955 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d99c3-d8eb-4cbf-8838-ae7d010396ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:58.701905 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d99c3-d8eb-4cbf-8838-ae7d010396ac admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:58.782927 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1361/2692] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:53:58 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:58.785581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a3152-f86b-4fb1-b77f-1c97d99e2531 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:58.795923 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a3152-f86b-4fb1-b77f-1c97d99e2531 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:58.851094 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0aee1a-a95b-4a08-bb9c-f01eaa45d119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:58.851759 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0aee1a-a95b-4a08-bb9c-f01eaa45d119 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:58.852050 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0aee1a-a95b-4a08-bb9c-f01eaa45d119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:58.852334 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0aee1a-a95b-4a08-bb9c-f01eaa45d119 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:58.853326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0aee1a-a95b-4a08-bb9c-f01eaa45d119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:58.855406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0aee1a-a95b-4a08-bb9c-f01eaa45d119 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:58.938737 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1332/2693] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:53:58 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:53:58.942796 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d99c3-d8eb-4cbf-8838-ae7d010396ac admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:58.953744 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d99c3-d8eb-4cbf-8838-ae7d010396ac admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:58.958390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced92cd-8f40-44e2-84ce-902e4fb982f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:58.959051 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced92cd-8f40-44e2-84ce-902e4fb982f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:58.959337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced92cd-8f40-44e2-84ce-902e4fb982f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:58.959618 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced92cd-8f40-44e2-84ce-902e4fb982f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:58.960344 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced92cd-8f40-44e2-84ce-902e4fb982f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:58.962020 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced92cd-8f40-44e2-84ce-902e4fb982f0 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:58.972984 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1362/2694] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:58.977065 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0aee1a-a95b-4a08-bb9c-f01eaa45d119 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:58.987560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0aee1a-a95b-4a08-bb9c-f01eaa45d119 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:58.992256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354af1ec-4ea9-4cee-80a9-80cd3d5e9687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:58.992816 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354af1ec-4ea9-4cee-80a9-80cd3d5e9687 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:58.993101 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354af1ec-4ea9-4cee-80a9-80cd3d5e9687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:58.993480 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354af1ec-4ea9-4cee-80a9-80cd3d5e9687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:58.994097 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354af1ec-4ea9-4cee-80a9-80cd3d5e9687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:58.995779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354af1ec-4ea9-4cee-80a9-80cd3d5e9687 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:59.006027 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1333/2695] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:53:59.011731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced92cd-8f40-44e2-84ce-902e4fb982f0 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:59.020926 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced92cd-8f40-44e2-84ce-902e4fb982f0 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:59.025829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6949c89-ab43-4f57-80c2-2f23de6b7e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:59.027322 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6949c89-ab43-4f57-80c2-2f23de6b7e0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:59.028870 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd46cab-ba3e-4289-a97e-e8c73adb4e9f None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:59.029023 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6949c89-ab43-4f57-80c2-2f23de6b7e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:59.029166 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd46cab-ba3e-4289-a97e-e8c73adb4e9f None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:59.029281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6949c89-ab43-4f57-80c2-2f23de6b7e0e admin admin] PATH_INFO: `/v3/projects/fd27cb3d6d844df7a8ccc9630336321b/users/ff7c17d8b83d4c8383305b7075eb6e84/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:59.029449 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd46cab-ba3e-4289-a97e-e8c73adb4e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:59.044015 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6949c89-ab43-4f57-80c2-2f23de6b7e0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd27cb3d6d844df7a8ccc9630336321b, user_id=ff7c17d8b83d4c8383305b7075eb6e84, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:59.046643 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6949c89-ab43-4f57-80c2-2f23de6b7e0e admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:59.046785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-ddd46cab-ba3e-4289-a97e-e8c73adb4e9f None None] MFA Rules not processed for user `2dcf17b8ad9349458e3328ebc2984902`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:59.048000 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ddd46cab-ba3e-4289-a97e-e8c73adb4e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:59.105503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1334/2696] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:53:59.109710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1363/2697] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:53:59 2022] PUT /identity/v3/projects/fd27cb3d6d844df7a8ccc9630336321b/users/ff7c17d8b83d4c8383305b7075eb6e84/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:53:59.120330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a481ea8-b5fe-47f7-97b9-307d1bc6726f None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:59.120897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a481ea8-b5fe-47f7-97b9-307d1bc6726f None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:59.121207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a481ea8-b5fe-47f7-97b9-307d1bc6726f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:59.148733 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-3a481ea8-b5fe-47f7-97b9-307d1bc6726f None None] MFA Rules not processed for user `ff7c17d8b83d4c8383305b7075eb6e84`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:53:59.149950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3a481ea8-b5fe-47f7-97b9-307d1bc6726f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:59.158817 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:53:59.216873 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1335/2698] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:53:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:53:59.228542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6949c89-ab43-4f57-80c2-2f23de6b7e0e admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:53:59.235904 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-b6949c89-ab43-4f57-80c2-2f23de6b7e0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:53:59.291623 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6949c89-ab43-4f57-80c2-2f23de6b7e0e admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:53:59.295652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8520c3-940d-4eed-b5b1-260125be5b8c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '983801bb18924fe4b3150693765eb13b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:53:59.296248 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8520c3-940d-4eed-b5b1-260125be5b8c service glance] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:53:59.296540 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8520c3-940d-4eed-b5b1-260125be5b8c service glance] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:53:59.296845 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8520c3-940d-4eed-b5b1-260125be5b8c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:53:59.305443 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8520c3-940d-4eed-b5b1-260125be5b8c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:53:59.306986 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8520c3-940d-4eed-b5b1-260125be5b8c service glance] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:53:59.316929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1364/2699] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:53:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:54:09.250460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=23 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:54:09.312804 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:54:20.822614 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0538342-dc38-48e1-9445-01796ac5ac56 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:20.823145 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0538342-dc38-48e1-9445-01796ac5ac56 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:20.823291 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0538342-dc38-48e1-9445-01796ac5ac56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:20.841708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-e0538342-dc38-48e1-9445-01796ac5ac56 None None] MFA Rules not processed for user `0a7c204a298c42f0bbc32fa9a6ce1f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:54:20.842922 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e0538342-dc38-48e1-9445-01796ac5ac56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:54:20.897735 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1336/2700] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 22:54:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:54:21.025424 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8520c3-940d-4eed-b5b1-260125be5b8c service glance] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:54:21.026975 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-4e8520c3-940d-4eed-b5b1-260125be5b8c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:54:21.098262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8520c3-940d-4eed-b5b1-260125be5b8c service glance] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:54:21.103645 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c488ce5-239b-47eb-bedb-fc9454cb9cf7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b321586cdb3d4d25904472aaa4adba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:54:21.104293 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c488ce5-239b-47eb-bedb-fc9454cb9cf7 service swift] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:21.104606 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c488ce5-239b-47eb-bedb-fc9454cb9cf7 service swift] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:21.104885 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c488ce5-239b-47eb-bedb-fc9454cb9cf7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:21.113977 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c488ce5-239b-47eb-bedb-fc9454cb9cf7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:54:21.115546 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c488ce5-239b-47eb-bedb-fc9454cb9cf7 service swift] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:54:21.123912 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1365/2701] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 22:54:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:54:30.890342 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=25 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:54:31.120536 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=32 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:54:31.302794 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0538342-dc38-48e1-9445-01796ac5ac56 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:54:31.376204 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0538342-dc38-48e1-9445-01796ac5ac56 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:54:31.382144 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510028fa-6ac9-49e9-b7d6-77c8d5ba9eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:54:31.383361 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510028fa-6ac9-49e9-b7d6-77c8d5ba9eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:31.383882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510028fa-6ac9-49e9-b7d6-77c8d5ba9eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:31.384225 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510028fa-6ac9-49e9-b7d6-77c8d5ba9eb5 admin admin] PATH_INFO: `/v3/users/ff7c17d8b83d4c8383305b7075eb6e84` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:31.385711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510028fa-6ac9-49e9-b7d6-77c8d5ba9eb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff7c17d8b83d4c8383305b7075eb6e84)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:54:31.387959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510028fa-6ac9-49e9-b7d6-77c8d5ba9eb5 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:54:31.568928 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-510028fa-6ac9-49e9-b7d6-77c8d5ba9eb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff7c17d8b83d4c8383305b7075eb6e84'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:54:31.585074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-510028fa-6ac9-49e9-b7d6-77c8d5ba9eb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff7c17d8b83d4c8383305b7075eb6e84'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:54:31.661695 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-510028fa-6ac9-49e9-b7d6-77c8d5ba9eb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff7c17d8b83d4c8383305b7075eb6e84'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:54:31.662987 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-510028fa-6ac9-49e9-b7d6-77c8d5ba9eb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff7c17d8b83d4c8383305b7075eb6e84'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:54:31.663729 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-510028fa-6ac9-49e9-b7d6-77c8d5ba9eb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff7c17d8b83d4c8383305b7075eb6e84'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:54:31.683027 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1337/2702] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:54:31 2022] DELETE /identity/v3/users/ff7c17d8b83d4c8383305b7075eb6e84 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:54:31.688553 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfeb1f-c135-4318-a600-8eb5d182a6f9 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:31.688894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfeb1f-c135-4318-a600-8eb5d182a6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:31.689195 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfeb1f-c135-4318-a600-8eb5d182a6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:31.708855 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-ebbfeb1f-c135-4318-a600-8eb5d182a6f9 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:54:31.712774 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ebbfeb1f-c135-4318-a600-8eb5d182a6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:54:31.779185 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1366/2703] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:54:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:54:31.789496 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510028fa-6ac9-49e9-b7d6-77c8d5ba9eb5 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:54:31.791278 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-510028fa-6ac9-49e9-b7d6-77c8d5ba9eb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:54:31.880226 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510028fa-6ac9-49e9-b7d6-77c8d5ba9eb5 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:54:31.884113 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2b9bc-483a-45b3-b42d-1204d71f0c49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:54:31.884819 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2b9bc-483a-45b3-b42d-1204d71f0c49 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:31.885126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2b9bc-483a-45b3-b42d-1204d71f0c49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:31.885386 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2b9bc-483a-45b3-b42d-1204d71f0c49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:31.897089 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2b9bc-483a-45b3-b42d-1204d71f0c49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:54:31.898743 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2b9bc-483a-45b3-b42d-1204d71f0c49 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:54:31.910244 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1338/2704] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:54:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:54:32.357678 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfeb1f-c135-4318-a600-8eb5d182a6f9 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:54:32.360515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ebbfeb1f-c135-4318-a600-8eb5d182a6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:54:32.400337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfeb1f-c135-4318-a600-8eb5d182a6f9 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:54:32.408775 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850894b9-5109-4dd2-9fb1-66eb4287c89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:54:32.410172 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850894b9-5109-4dd2-9fb1-66eb4287c89d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:32.410635 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850894b9-5109-4dd2-9fb1-66eb4287c89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:32.410851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850894b9-5109-4dd2-9fb1-66eb4287c89d admin admin] PATH_INFO: `/v3/projects/fd27cb3d6d844df7a8ccc9630336321b` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:32.412120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850894b9-5109-4dd2-9fb1-66eb4287c89d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd27cb3d6d844df7a8ccc9630336321b)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:54:32.414255 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850894b9-5109-4dd2-9fb1-66eb4287c89d admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:54:32.518341 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-850894b9-5109-4dd2-9fb1-66eb4287c89d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd27cb3d6d844df7a8ccc9630336321b', 'request_id': 'req-850894b9-5109-4dd2-9fb1-66eb4287c89d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:54:32.525253 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-850894b9-5109-4dd2-9fb1-66eb4287c89d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd27cb3d6d844df7a8ccc9630336321b', 'request_id': 'req-850894b9-5109-4dd2-9fb1-66eb4287c89d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:54:32.531497 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-850894b9-5109-4dd2-9fb1-66eb4287c89d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd27cb3d6d844df7a8ccc9630336321b', 'request_id': 'req-850894b9-5109-4dd2-9fb1-66eb4287c89d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:54:32.607453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-850894b9-5109-4dd2-9fb1-66eb4287c89d admin admin] The token cache is being invalidate because project fd27cb3d6d844df7a8ccc9630336321b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:54:32.608148 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-850894b9-5109-4dd2-9fb1-66eb4287c89d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:54:32.608853 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-850894b9-5109-4dd2-9fb1-66eb4287c89d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:54:32.611615 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1367/2705] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:54:32 2022] DELETE /identity/v3/projects/fd27cb3d6d844df7a8ccc9630336321b => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:54:32.617366 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d8d01-540b-49cd-9ae4-47d09a2afa5a None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:32.617738 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d8d01-540b-49cd-9ae4-47d09a2afa5a None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:32.618124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d8d01-540b-49cd-9ae4-47d09a2afa5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:32.638435 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-7f0d8d01-540b-49cd-9ae4-47d09a2afa5a None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:54:32.639708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-7f0d8d01-540b-49cd-9ae4-47d09a2afa5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:54:32.661534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:54:32.709557 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1339/2706] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:54:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:54:32.719237 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc600719-db30-4fe8-a844-2071b031d5fa None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:32.719759 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc600719-db30-4fe8-a844-2071b031d5fa None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:32.720272 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc600719-db30-4fe8-a844-2071b031d5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:32.741642 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-cc600719-db30-4fe8-a844-2071b031d5fa None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:54:32.742925 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-cc600719-db30-4fe8-a844-2071b031d5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:54:32.752355 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1368/2707] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:54:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:54:32.757835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d8d01-540b-49cd-9ae4-47d09a2afa5a None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:54:32.768866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d8d01-540b-49cd-9ae4-47d09a2afa5a None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:54:32.777336 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e090c-97b1-471f-b7fd-e509e379309e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:54:32.778038 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e090c-97b1-471f-b7fd-e509e379309e admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:32.778325 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e090c-97b1-471f-b7fd-e509e379309e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:32.778635 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e090c-97b1-471f-b7fd-e509e379309e admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:32.779601 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e090c-97b1-471f-b7fd-e509e379309e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:54:32.779915 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e090c-97b1-471f-b7fd-e509e379309e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:54:32.782301 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e090c-97b1-471f-b7fd-e509e379309e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:54:32.796421 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1340/2708] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:54:32.802611 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc600719-db30-4fe8-a844-2071b031d5fa None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:54:32.812266 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc600719-db30-4fe8-a844-2071b031d5fa None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:54:32.817274 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba8142-1839-4ec8-b3a0-00652c40206b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:54:32.817981 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba8142-1839-4ec8-b3a0-00652c40206b admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:32.818264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba8142-1839-4ec8-b3a0-00652c40206b admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:32.818544 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba8142-1839-4ec8-b3a0-00652c40206b admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:32.819265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba8142-1839-4ec8-b3a0-00652c40206b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:54:32.821211 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba8142-1839-4ec8-b3a0-00652c40206b admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:54:33.380781 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1369/2709] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:54:32 2022] POST /identity/v3/projects => generated 363 bytes in 579 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:54:33.385062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e090c-97b1-471f-b7fd-e509e379309e admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:54:33.395790 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e090c-97b1-471f-b7fd-e509e379309e admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:54:33.456024 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c1b7f-8149-4f27-b94b-0739d50afdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:54:33.456743 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c1b7f-8149-4f27-b94b-0739d50afdf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:33.457026 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c1b7f-8149-4f27-b94b-0739d50afdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:33.457307 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c1b7f-8149-4f27-b94b-0739d50afdf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:33.458001 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81c1b7f-8149-4f27-b94b-0739d50afdf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:54:33.459946 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81c1b7f-8149-4f27-b94b-0739d50afdf8 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:54:33.582501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1341/2710] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:54:33 2022] POST /identity/v3/users => generated 282 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:54:33.583674 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba8142-1839-4ec8-b3a0-00652c40206b admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:54:33.595075 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba8142-1839-4ec8-b3a0-00652c40206b admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:54:33.599998 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510e74a-a3ca-4f7f-b7d2-e95e7557514b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:54:33.600707 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510e74a-a3ca-4f7f-b7d2-e95e7557514b admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:33.601120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510e74a-a3ca-4f7f-b7d2-e95e7557514b admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:33.601689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510e74a-a3ca-4f7f-b7d2-e95e7557514b admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:33.602532 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510e74a-a3ca-4f7f-b7d2-e95e7557514b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:54:33.604222 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510e74a-a3ca-4f7f-b7d2-e95e7557514b admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:54:33.620636 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1370/2711] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:54:33.622922 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c1b7f-8149-4f27-b94b-0739d50afdf8 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:54:33.637041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c1b7f-8149-4f27-b94b-0739d50afdf8 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:54:33.642566 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51819488-0d92-4bc7-a2f3-3f75121e13f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:54:33.643326 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51819488-0d92-4bc7-a2f3-3f75121e13f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:33.643679 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51819488-0d92-4bc7-a2f3-3f75121e13f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:33.644023 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51819488-0d92-4bc7-a2f3-3f75121e13f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:33.644786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51819488-0d92-4bc7-a2f3-3f75121e13f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:54:33.647133 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51819488-0d92-4bc7-a2f3-3f75121e13f4 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:54:33.660793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1342/2712] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:54:33.664872 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510e74a-a3ca-4f7f-b7d2-e95e7557514b admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:54:33.676369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510e74a-a3ca-4f7f-b7d2-e95e7557514b admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:54:33.684889 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb902d26-7c29-466f-9d39-abe581da9ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:54:33.686130 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb902d26-7c29-466f-9d39-abe581da9ee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:33.686601 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb902d26-7c29-466f-9d39-abe581da9ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:33.686881 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb902d26-7c29-466f-9d39-abe581da9ee1 admin admin] PATH_INFO: `/v3/projects/539360fb4353463cb58c140d779e3b97/users/fb86cf62c071470d85d9daa70084fc19/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:33.705206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb902d26-7c29-466f-9d39-abe581da9ee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=539360fb4353463cb58c140d779e3b97, user_id=fb86cf62c071470d85d9daa70084fc19, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:54:33.707911 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb902d26-7c29-466f-9d39-abe581da9ee1 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:54:33.813731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1371/2713] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:54:33 2022] PUT /identity/v3/projects/539360fb4353463cb58c140d779e3b97/users/fb86cf62c071470d85d9daa70084fc19/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:54:33.878563 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a968c63-0d7f-40f6-85ec-2e9031d827d5 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:33.878857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a968c63-0d7f-40f6-85ec-2e9031d827d5 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:33.879139 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a968c63-0d7f-40f6-85ec-2e9031d827d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:33.916236 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-9a968c63-0d7f-40f6-85ec-2e9031d827d5 None None] MFA Rules not processed for user `fb86cf62c071470d85d9daa70084fc19`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:54:33.917903 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-9a968c63-0d7f-40f6-85ec-2e9031d827d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:54:34.000611 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1343/2714] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:54:33 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:54:34.012866 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb902d26-7c29-466f-9d39-abe581da9ee1 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:54:34.014643 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-bb902d26-7c29-466f-9d39-abe581da9ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:54:34.044296 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:54:34.086362 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb902d26-7c29-466f-9d39-abe581da9ee1 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:54:34.090332 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e3690d-b99e-47f0-ac90-78701b885861 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:54:34.091202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3690d-b99e-47f0-ac90-78701b885861 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:34.091670 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3690d-b99e-47f0-ac90-78701b885861 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:34.091951 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3690d-b99e-47f0-ac90-78701b885861 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:34.104977 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e3690d-b99e-47f0-ac90-78701b885861 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:54:34.114189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e3690d-b99e-47f0-ac90-78701b885861 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:54:34.119504 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1372/2715] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:54:34 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:54:39.168535 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e8739-ee76-4832-8d87-30fb890a8c24 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:39.169006 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e8739-ee76-4832-8d87-30fb890a8c24 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:39.169126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e8739-ee76-4832-8d87-30fb890a8c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:39.186385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-c73e8739-ee76-4832-8d87-30fb890a8c24 None None] MFA Rules not processed for user `0a7c204a298c42f0bbc32fa9a6ce1f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:54:39.187576 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c73e8739-ee76-4832-8d87-30fb890a8c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:54:39.199537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:54:39.243034 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1344/2716] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 23 22:54:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:54:39.255237 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e3690d-b99e-47f0-ac90-78701b885861 service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:54:39.256720 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a8e3690d-b99e-47f0-ac90-78701b885861 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:54:39.326850 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e3690d-b99e-47f0-ac90-78701b885861 service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:54:39.332080 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd688e-7fc9-4942-9781-d8f011862cb4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b321586cdb3d4d25904472aaa4adba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:54:39.332757 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd688e-7fc9-4942-9781-d8f011862cb4 service swift] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:39.333045 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd688e-7fc9-4942-9781-d8f011862cb4 service swift] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:39.333322 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd688e-7fc9-4942-9781-d8f011862cb4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:39.341221 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd688e-7fc9-4942-9781-d8f011862cb4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:54:39.342822 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd688e-7fc9-4942-9781-d8f011862cb4 service swift] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:54:39.351507 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1373/2717] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 22:54:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:54:42.414900 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e8739-ee76-4832-8d87-30fb890a8c24 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:54:42.416769 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-c73e8739-ee76-4832-8d87-30fb890a8c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:54:42.504280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e8739-ee76-4832-8d87-30fb890a8c24 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:54:42.508841 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676399b9-b530-47a8-b5e9-4f5e952bfe78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:54:42.509723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676399b9-b530-47a8-b5e9-4f5e952bfe78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:42.510011 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676399b9-b530-47a8-b5e9-4f5e952bfe78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:42.510291 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676399b9-b530-47a8-b5e9-4f5e952bfe78 admin admin] PATH_INFO: `/v3/users/2dcf17b8ad9349458e3328ebc2984902` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:42.511508 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676399b9-b530-47a8-b5e9-4f5e952bfe78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dcf17b8ad9349458e3328ebc2984902)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:54:42.513925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676399b9-b530-47a8-b5e9-4f5e952bfe78 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:54:42.708545 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-676399b9-b530-47a8-b5e9-4f5e952bfe78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dcf17b8ad9349458e3328ebc2984902'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:54:42.720484 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-676399b9-b530-47a8-b5e9-4f5e952bfe78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dcf17b8ad9349458e3328ebc2984902'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:54:42.800966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-676399b9-b530-47a8-b5e9-4f5e952bfe78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dcf17b8ad9349458e3328ebc2984902'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:54:42.801926 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-676399b9-b530-47a8-b5e9-4f5e952bfe78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dcf17b8ad9349458e3328ebc2984902'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:54:42.802734 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-676399b9-b530-47a8-b5e9-4f5e952bfe78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dcf17b8ad9349458e3328ebc2984902'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:54:42.816599 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1345/2718] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:54:42 2022] DELETE /identity/v3/users/2dcf17b8ad9349458e3328ebc2984902 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:54:43.069613 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd688e-7fc9-4942-9781-d8f011862cb4 service swift] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:54:43.087320 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd688e-7fc9-4942-9781-d8f011862cb4 service swift] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:54:43.168453 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb9254-4f93-41a4-af16-414617e9f30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:54:43.169490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb9254-4f93-41a4-af16-414617e9f30a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:43.169813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb9254-4f93-41a4-af16-414617e9f30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:43.170091 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb9254-4f93-41a4-af16-414617e9f30a admin admin] PATH_INFO: `/v3/projects/7cee5e6784454293bc02e0e595d58b22` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:43.171280 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb9254-4f93-41a4-af16-414617e9f30a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cee5e6784454293bc02e0e595d58b22)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:54:43.173274 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb9254-4f93-41a4-af16-414617e9f30a admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:54:43.333602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6edb9254-4f93-41a4-af16-414617e9f30a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cee5e6784454293bc02e0e595d58b22', 'request_id': 'req-6edb9254-4f93-41a4-af16-414617e9f30a'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:54:43.339710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6edb9254-4f93-41a4-af16-414617e9f30a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cee5e6784454293bc02e0e595d58b22', 'request_id': 'req-6edb9254-4f93-41a4-af16-414617e9f30a'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:54:43.340956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:54:43.345694 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6edb9254-4f93-41a4-af16-414617e9f30a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cee5e6784454293bc02e0e595d58b22', 'request_id': 'req-6edb9254-4f93-41a4-af16-414617e9f30a'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:54:43.443095 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6edb9254-4f93-41a4-af16-414617e9f30a admin admin] The token cache is being invalidate because project 7cee5e6784454293bc02e0e595d58b22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:54:43.443305 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6edb9254-4f93-41a4-af16-414617e9f30a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:54:43.443936 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-6edb9254-4f93-41a4-af16-414617e9f30a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:54:43.445526 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1374/2719] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:54:43 2022] DELETE /identity/v3/projects/7cee5e6784454293bc02e0e595d58b22 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:54:52.812515 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=70,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:54:53.211941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec01e15-dc87-4238-bca6-78009508c8e9 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:54:53.212648 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec01e15-dc87-4238-bca6-78009508c8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:54:53.213503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec01e15-dc87-4238-bca6-78009508c8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:54:53.236666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-dec01e15-dc87-4238-bca6-78009508c8e9 None None] MFA Rules not processed for user `92ce6158277146228cb13194e9d7b04b`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:54:53.237930 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-dec01e15-dc87-4238-bca6-78009508c8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:54:53.303650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1346/2720] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:54:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:54:53.477941 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:DELETE=3,keystone:SELECT=4,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:55:00.989794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb9254-4f93-41a4-af16-414617e9f30a admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:55:00.991336 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6edb9254-4f93-41a4-af16-414617e9f30a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:55:01.060279 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb9254-4f93-41a4-af16-414617e9f30a admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:55:01.063898 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c829dc-1698-408f-9dbb-013fdec06ae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:55:01.064551 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c829dc-1698-408f-9dbb-013fdec06ae0 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:55:01.064882 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c829dc-1698-408f-9dbb-013fdec06ae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:55:01.065167 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c829dc-1698-408f-9dbb-013fdec06ae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:55:01.067918 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-86c829dc-1698-408f-9dbb-013fdec06ae0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:55:01.149622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c829dc-1698-408f-9dbb-013fdec06ae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:55:01.151074 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c829dc-1698-408f-9dbb-013fdec06ae0 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:55:01.163306 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1375/2721] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:55:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:55:03.298767 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=25 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:55:07.545244 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec01e15-dc87-4238-bca6-78009508c8e9 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:55:07.546734 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-dec01e15-dc87-4238-bca6-78009508c8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:55:07.567957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec01e15-dc87-4238-bca6-78009508c8e9 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:55:07.572272 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2111e06-fed2-4217-a2f4-4829df3e1345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:55:07.573447 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2111e06-fed2-4217-a2f4-4829df3e1345 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:55:07.574029 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2111e06-fed2-4217-a2f4-4829df3e1345 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:55:07.574473 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2111e06-fed2-4217-a2f4-4829df3e1345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:55:07.576623 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-b2111e06-fed2-4217-a2f4-4829df3e1345 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:55:07.685375 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2111e06-fed2-4217-a2f4-4829df3e1345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:55:07.688277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2111e06-fed2-4217-a2f4-4829df3e1345 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:55:07.710756 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1347/2722] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:55:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:55:09.140112 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c829dc-1698-408f-9dbb-013fdec06ae0 service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:55:09.142566 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-86c829dc-1698-408f-9dbb-013fdec06ae0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:55:09.159811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c829dc-1698-408f-9dbb-013fdec06ae0 service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:55:09.164977 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c120980-5ce3-49df-bed5-5d80dcafeaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:55:09.165889 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c120980-5ce3-49df-bed5-5d80dcafeaff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:55:09.166416 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c120980-5ce3-49df-bed5-5d80dcafeaff admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:55:09.166692 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c120980-5ce3-49df-bed5-5d80dcafeaff admin admin] PATH_INFO: `/v3/users/84a5fb262284400591cf29dd13771ee7` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:55:09.167994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c120980-5ce3-49df-bed5-5d80dcafeaff admin admin] RBAC: Authorizing `identity:delete_user(user_id=84a5fb262284400591cf29dd13771ee7)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:55:09.169972 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c120980-5ce3-49df-bed5-5d80dcafeaff admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:55:09.405572 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-8c120980-5ce3-49df-bed5-5d80dcafeaff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84a5fb262284400591cf29dd13771ee7'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:09.423505 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-8c120980-5ce3-49df-bed5-5d80dcafeaff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84a5fb262284400591cf29dd13771ee7'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:09.423505 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-8c120980-5ce3-49df-bed5-5d80dcafeaff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84a5fb262284400591cf29dd13771ee7'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:09.423505 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-8c120980-5ce3-49df-bed5-5d80dcafeaff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84a5fb262284400591cf29dd13771ee7'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:09.431740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-8c120980-5ce3-49df-bed5-5d80dcafeaff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84a5fb262284400591cf29dd13771ee7'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:09.528617 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1376/2723] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:55:09 2022] DELETE /identity/v3/users/84a5fb262284400591cf29dd13771ee7 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:55:09.780885 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2111e06-fed2-4217-a2f4-4829df3e1345 service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:55:09.795621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2111e06-fed2-4217-a2f4-4829df3e1345 service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:55:09.867851 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992a0ef-5d26-4968-860c-7f426bbbdd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:55:09.868849 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992a0ef-5d26-4968-860c-7f426bbbdd54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:55:09.869201 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992a0ef-5d26-4968-860c-7f426bbbdd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:55:09.869690 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992a0ef-5d26-4968-860c-7f426bbbdd54 admin admin] PATH_INFO: `/v3/projects/6fdc472916934fac985e88fb6a61290b` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:55:09.870986 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992a0ef-5d26-4968-860c-7f426bbbdd54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fdc472916934fac985e88fb6a61290b)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:55:09.873163 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992a0ef-5d26-4968-860c-7f426bbbdd54 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:55:09.999792 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7992a0ef-5d26-4968-860c-7f426bbbdd54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fdc472916934fac985e88fb6a61290b', 'request_id': 'req-7992a0ef-5d26-4968-860c-7f426bbbdd54'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:10.059449 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7992a0ef-5d26-4968-860c-7f426bbbdd54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fdc472916934fac985e88fb6a61290b', 'request_id': 'req-7992a0ef-5d26-4968-860c-7f426bbbdd54'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:10.067497 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7992a0ef-5d26-4968-860c-7f426bbbdd54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fdc472916934fac985e88fb6a61290b', 'request_id': 'req-7992a0ef-5d26-4968-860c-7f426bbbdd54'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:10.079779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7992a0ef-5d26-4968-860c-7f426bbbdd54 admin admin] The token cache is being invalidate because project 6fdc472916934fac985e88fb6a61290b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:55:10.080190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7992a0ef-5d26-4968-860c-7f426bbbdd54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:10.080998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7992a0ef-5d26-4968-860c-7f426bbbdd54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:10.083642 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1348/2724] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:55:09 2022] DELETE /identity/v3/projects/6fdc472916934fac985e88fb6a61290b => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:55:10.087852 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c120980-5ce3-49df-bed5-5d80dcafeaff admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:55:10.088953 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-8c120980-5ce3-49df-bed5-5d80dcafeaff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:55:10.108542 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c120980-5ce3-49df-bed5-5d80dcafeaff admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:55:10.127699 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:55:10.168598 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdbdc1-1a9f-4cb6-8f28-1667e10bce0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:55:10.169499 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdbdc1-1a9f-4cb6-8f28-1667e10bce0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:55:10.169786 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdbdc1-1a9f-4cb6-8f28-1667e10bce0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:55:10.170065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdbdc1-1a9f-4cb6-8f28-1667e10bce0f admin admin] PATH_INFO: `/v3/users/e2e0900058f7442fa48c48903781317c` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:55:10.171258 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfdbdc1-1a9f-4cb6-8f28-1667e10bce0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2e0900058f7442fa48c48903781317c)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:55:10.173406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfdbdc1-1a9f-4cb6-8f28-1667e10bce0f admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:55:10.386711 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-ddfdbdc1-1a9f-4cb6-8f28-1667e10bce0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2e0900058f7442fa48c48903781317c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:10.398736 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-ddfdbdc1-1a9f-4cb6-8f28-1667e10bce0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2e0900058f7442fa48c48903781317c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:10.399418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-ddfdbdc1-1a9f-4cb6-8f28-1667e10bce0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2e0900058f7442fa48c48903781317c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:10.400022 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-ddfdbdc1-1a9f-4cb6-8f28-1667e10bce0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2e0900058f7442fa48c48903781317c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:10.406945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-ddfdbdc1-1a9f-4cb6-8f28-1667e10bce0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2e0900058f7442fa48c48903781317c'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:10.508619 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1377/2725] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:55:10 2022] DELETE /identity/v3/users/e2e0900058f7442fa48c48903781317c => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:55:10.751985 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992a0ef-5d26-4968-860c-7f426bbbdd54 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:55:10.753961 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-7992a0ef-5d26-4968-860c-7f426bbbdd54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:55:10.769528 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992a0ef-5d26-4968-860c-7f426bbbdd54 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:55:10.781887 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:55:10.829328 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528c540-fb79-48fb-9a57-19b2d18f87b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:55:10.830194 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528c540-fb79-48fb-9a57-19b2d18f87b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:55:10.830474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528c540-fb79-48fb-9a57-19b2d18f87b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:55:10.830750 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528c540-fb79-48fb-9a57-19b2d18f87b3 admin admin] PATH_INFO: `/v3/projects/a2afecbf44e543fd89fafe08a7ec7a01` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:55:10.832220 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528c540-fb79-48fb-9a57-19b2d18f87b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2afecbf44e543fd89fafe08a7ec7a01)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:55:10.834168 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528c540-fb79-48fb-9a57-19b2d18f87b3 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:55:10.936101 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6528c540-fb79-48fb-9a57-19b2d18f87b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2afecbf44e543fd89fafe08a7ec7a01', 'request_id': 'req-6528c540-fb79-48fb-9a57-19b2d18f87b3'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:10.998901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6528c540-fb79-48fb-9a57-19b2d18f87b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2afecbf44e543fd89fafe08a7ec7a01', 'request_id': 'req-6528c540-fb79-48fb-9a57-19b2d18f87b3'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:11.005845 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6528c540-fb79-48fb-9a57-19b2d18f87b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2afecbf44e543fd89fafe08a7ec7a01', 'request_id': 'req-6528c540-fb79-48fb-9a57-19b2d18f87b3'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:11.016398 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6528c540-fb79-48fb-9a57-19b2d18f87b3 admin admin] The token cache is being invalidate because project a2afecbf44e543fd89fafe08a7ec7a01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:55:11.016728 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6528c540-fb79-48fb-9a57-19b2d18f87b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:11.017397 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-6528c540-fb79-48fb-9a57-19b2d18f87b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:11.018927 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1349/2726] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:55:10 2022] DELETE /identity/v3/projects/a2afecbf44e543fd89fafe08a7ec7a01 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:55:13.668097 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdbdc1-1a9f-4cb6-8f28-1667e10bce0f admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:55:13.669557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ddfdbdc1-1a9f-4cb6-8f28-1667e10bce0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:55:13.682362 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdbdc1-1a9f-4cb6-8f28-1667e10bce0f admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:55:13.739625 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b60966-a217-4eac-b72f-fe3d3edb1320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:55:13.740567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b60966-a217-4eac-b72f-fe3d3edb1320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:55:13.740877 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b60966-a217-4eac-b72f-fe3d3edb1320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:55:13.743917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b60966-a217-4eac-b72f-fe3d3edb1320 admin admin] PATH_INFO: `/v3/users/fb86cf62c071470d85d9daa70084fc19` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:55:13.743917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b60966-a217-4eac-b72f-fe3d3edb1320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb86cf62c071470d85d9daa70084fc19)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:55:13.744400 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b60966-a217-4eac-b72f-fe3d3edb1320 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:55:13.892695 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:55:13.895915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-95b60966-a217-4eac-b72f-fe3d3edb1320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb86cf62c071470d85d9daa70084fc19'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:13.911516 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-95b60966-a217-4eac-b72f-fe3d3edb1320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb86cf62c071470d85d9daa70084fc19'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:13.911516 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-95b60966-a217-4eac-b72f-fe3d3edb1320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb86cf62c071470d85d9daa70084fc19'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:13.911516 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-95b60966-a217-4eac-b72f-fe3d3edb1320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb86cf62c071470d85d9daa70084fc19'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:13.916053 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-95b60966-a217-4eac-b72f-fe3d3edb1320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb86cf62c071470d85d9daa70084fc19'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:13.993111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1378/2727] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:55:13 2022] DELETE /identity/v3/users/fb86cf62c071470d85d9daa70084fc19 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:55:13.997797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69ba02-4d4c-4350-b9f6-90b65c501a69 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:55:13.998311 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69ba02-4d4c-4350-b9f6-90b65c501a69 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:55:13.998435 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69ba02-4d4c-4350-b9f6-90b65c501a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:55:14.016213 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-9d69ba02-4d4c-4350-b9f6-90b65c501a69 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:55:14.017663 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-9d69ba02-4d4c-4350-b9f6-90b65c501a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:55:14.076025 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1350/2728] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:55:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:55:14.086055 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b60966-a217-4eac-b72f-fe3d3edb1320 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:55:14.087602 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-95b60966-a217-4eac-b72f-fe3d3edb1320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:55:14.145497 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b60966-a217-4eac-b72f-fe3d3edb1320 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:55:14.149244 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecc1d1-6b00-4b2f-8289-e2b2cb0c2656 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:55:14.149894 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecc1d1-6b00-4b2f-8289-e2b2cb0c2656 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:55:14.150182 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecc1d1-6b00-4b2f-8289-e2b2cb0c2656 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:55:14.150465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecc1d1-6b00-4b2f-8289-e2b2cb0c2656 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:55:14.159486 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ecc1d1-6b00-4b2f-8289-e2b2cb0c2656 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:55:14.161024 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ecc1d1-6b00-4b2f-8289-e2b2cb0c2656 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:55:14.170702 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1379/2729] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:55:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:55:14.589862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69ba02-4d4c-4350-b9f6-90b65c501a69 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:55:14.591454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-9d69ba02-4d4c-4350-b9f6-90b65c501a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:55:14.606360 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69ba02-4d4c-4350-b9f6-90b65c501a69 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:55:14.611562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ba815-26af-4666-9a47-a0dda09685e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:55:14.612697 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ba815-26af-4666-9a47-a0dda09685e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:55:14.613043 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ba815-26af-4666-9a47-a0dda09685e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:55:14.614066 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ba815-26af-4666-9a47-a0dda09685e7 admin admin] PATH_INFO: `/v3/projects/539360fb4353463cb58c140d779e3b97` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:55:14.615111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ba815-26af-4666-9a47-a0dda09685e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=539360fb4353463cb58c140d779e3b97)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:55:14.617051 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ba815-26af-4666-9a47-a0dda09685e7 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:55:14.713428 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:55:14.719212 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3b0ba815-26af-4666-9a47-a0dda09685e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '539360fb4353463cb58c140d779e3b97', 'request_id': 'req-3b0ba815-26af-4666-9a47-a0dda09685e7'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:14.782026 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3b0ba815-26af-4666-9a47-a0dda09685e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '539360fb4353463cb58c140d779e3b97', 'request_id': 'req-3b0ba815-26af-4666-9a47-a0dda09685e7'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:14.788901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3b0ba815-26af-4666-9a47-a0dda09685e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '539360fb4353463cb58c140d779e3b97', 'request_id': 'req-3b0ba815-26af-4666-9a47-a0dda09685e7'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:14.799859 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3b0ba815-26af-4666-9a47-a0dda09685e7 admin admin] The token cache is being invalidate because project 539360fb4353463cb58c140d779e3b97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:55:14.800275 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3b0ba815-26af-4666-9a47-a0dda09685e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:14.800959 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3b0ba815-26af-4666-9a47-a0dda09685e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:55:14.803083 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1351/2730] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:55:14 2022] DELETE /identity/v3/projects/539360fb4353463cb58c140d779e3b97 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:55:24.166375 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:DELETE=4,keystone:SELECT=39,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:55:24.863455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=9,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:55:46.954999 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecc1d1-6b00-4b2f-8289-e2b2cb0c2656 service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:55:46.956526 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-04ecc1d1-6b00-4b2f-8289-e2b2cb0c2656 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:55:47.013258 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecc1d1-6b00-4b2f-8289-e2b2cb0c2656 service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:55:47.060322 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53804673-495c-47c4-a082-a15f97c721ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:55:47.060907 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53804673-495c-47c4-a082-a15f97c721ef service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:55:47.061194 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53804673-495c-47c4-a082-a15f97c721ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:55:47.061472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53804673-495c-47c4-a082-a15f97c721ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:55:47.063257 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-53804673-495c-47c4-a082-a15f97c721ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:55:47.130956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53804673-495c-47c4-a082-a15f97c721ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:55:47.133056 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53804673-495c-47c4-a082-a15f97c721ef service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:55:47.143828 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1380/2731] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:55:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:55:57.138942 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=74 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:11.893390 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ba815-26af-4666-9a47-a0dda09685e7 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:11.894975 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-3b0ba815-26af-4666-9a47-a0dda09685e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:11.911669 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ba815-26af-4666-9a47-a0dda09685e7 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:11.917229 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86627d4-47a7-4ff3-9e1f-eaee8bbb880f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:11.917877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86627d4-47a7-4ff3-9e1f-eaee8bbb880f service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:11.918162 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86627d4-47a7-4ff3-9e1f-eaee8bbb880f service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:11.918560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86627d4-47a7-4ff3-9e1f-eaee8bbb880f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:11.920320 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e86627d4-47a7-4ff3-9e1f-eaee8bbb880f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:11.937957 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86627d4-47a7-4ff3-9e1f-eaee8bbb880f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:11.938251 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86627d4-47a7-4ff3-9e1f-eaee8bbb880f service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:11.951457 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1352/2732] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:56:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:56:17.237104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53804673-495c-47c4-a082-a15f97c721ef service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:17.238497 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-53804673-495c-47c4-a082-a15f97c721ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:17.249158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53804673-495c-47c4-a082-a15f97c721ef service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:17.252739 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a54bd2-396d-4858-8a24-0450f7bd6d1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:17.253401 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a54bd2-396d-4858-8a24-0450f7bd6d1d service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:17.253685 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a54bd2-396d-4858-8a24-0450f7bd6d1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:17.253962 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a54bd2-396d-4858-8a24-0450f7bd6d1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:17.256159 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-64a54bd2-396d-4858-8a24-0450f7bd6d1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:17.314946 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a54bd2-396d-4858-8a24-0450f7bd6d1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:17.316374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a54bd2-396d-4858-8a24-0450f7bd6d1d service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:17.325872 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1381/2733] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:56:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:56:18.740896 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86627d4-47a7-4ff3-9e1f-eaee8bbb880f service nova] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:18.744427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e86627d4-47a7-4ff3-9e1f-eaee8bbb880f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:18.797695 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86627d4-47a7-4ff3-9e1f-eaee8bbb880f service nova] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:18.801278 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09fece-d712-4306-9c1d-fa5e0d74a63e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:18.801909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09fece-d712-4306-9c1d-fa5e0d74a63e service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:18.802194 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09fece-d712-4306-9c1d-fa5e0d74a63e service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:18.802520 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09fece-d712-4306-9c1d-fa5e0d74a63e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:18.804357 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-6c09fece-d712-4306-9c1d-fa5e0d74a63e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:18.866855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09fece-d712-4306-9c1d-fa5e0d74a63e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:18.868287 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09fece-d712-4306-9c1d-fa5e0d74a63e service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:18.877871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1353/2734] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:56:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:56:22.182842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a54bd2-396d-4858-8a24-0450f7bd6d1d service cinder] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:22.184605 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-64a54bd2-396d-4858-8a24-0450f7bd6d1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:22.199794 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a54bd2-396d-4858-8a24-0450f7bd6d1d service cinder] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:22.204413 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b5b6d-3b25-46c8-a73c-e0aa3e3c4324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:22.205324 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b5b6d-3b25-46c8-a73c-e0aa3e3c4324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:22.205609 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b5b6d-3b25-46c8-a73c-e0aa3e3c4324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:22.205885 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b5b6d-3b25-46c8-a73c-e0aa3e3c4324 admin admin] PATH_INFO: `/v3/users/92ce6158277146228cb13194e9d7b04b` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:22.207065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b5b6d-3b25-46c8-a73c-e0aa3e3c4324 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92ce6158277146228cb13194e9d7b04b)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:22.209395 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b5b6d-3b25-46c8-a73c-e0aa3e3c4324 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:22.331724 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-000b5b6d-3b25-46c8-a73c-e0aa3e3c4324 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92ce6158277146228cb13194e9d7b04b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:22.343296 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-000b5b6d-3b25-46c8-a73c-e0aa3e3c4324 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92ce6158277146228cb13194e9d7b04b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:22.343954 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-000b5b6d-3b25-46c8-a73c-e0aa3e3c4324 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92ce6158277146228cb13194e9d7b04b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:22.344551 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-000b5b6d-3b25-46c8-a73c-e0aa3e3c4324 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92ce6158277146228cb13194e9d7b04b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:22.352162 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-000b5b6d-3b25-46c8-a73c-e0aa3e3c4324 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92ce6158277146228cb13194e9d7b04b'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:22.419069 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1382/2735] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:56:22 2022] DELETE /identity/v3/users/92ce6158277146228cb13194e9d7b04b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:22.656604 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09fece-d712-4306-9c1d-fa5e0d74a63e service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:22.667091 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09fece-d712-4306-9c1d-fa5e0d74a63e service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:22.724989 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980ca7a-e95f-45df-a3a8-8dd681bcd6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:22.726012 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980ca7a-e95f-45df-a3a8-8dd681bcd6b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:22.726299 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980ca7a-e95f-45df-a3a8-8dd681bcd6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:22.726584 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980ca7a-e95f-45df-a3a8-8dd681bcd6b8 admin admin] PATH_INFO: `/v3/projects/73d61a582ec24f99aaf018e55f1f049b` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:22.727778 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980ca7a-e95f-45df-a3a8-8dd681bcd6b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73d61a582ec24f99aaf018e55f1f049b)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:22.729724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980ca7a-e95f-45df-a3a8-8dd681bcd6b8 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:22.824858 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:22.832029 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3980ca7a-e95f-45df-a3a8-8dd681bcd6b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73d61a582ec24f99aaf018e55f1f049b', 'request_id': 'req-3980ca7a-e95f-45df-a3a8-8dd681bcd6b8'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:22.913798 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3980ca7a-e95f-45df-a3a8-8dd681bcd6b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73d61a582ec24f99aaf018e55f1f049b', 'request_id': 'req-3980ca7a-e95f-45df-a3a8-8dd681bcd6b8'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:22.920685 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3980ca7a-e95f-45df-a3a8-8dd681bcd6b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73d61a582ec24f99aaf018e55f1f049b', 'request_id': 'req-3980ca7a-e95f-45df-a3a8-8dd681bcd6b8'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:22.931365 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3980ca7a-e95f-45df-a3a8-8dd681bcd6b8 admin admin] The token cache is being invalidate because project 73d61a582ec24f99aaf018e55f1f049b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:56:22.931655 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3980ca7a-e95f-45df-a3a8-8dd681bcd6b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:22.932374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-3980ca7a-e95f-45df-a3a8-8dd681bcd6b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:22.933969 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1354/2736] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:56:22 2022] DELETE /identity/v3/projects/73d61a582ec24f99aaf018e55f1f049b => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:22.938925 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe52838-8df3-4f9f-9867-9f2c61f60146 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:22.939222 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe52838-8df3-4f9f-9867-9f2c61f60146 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:22.939509 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe52838-8df3-4f9f-9867-9f2c61f60146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:22.958157 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-dbe52838-8df3-4f9f-9867-9f2c61f60146 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:22.960448 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-dbe52838-8df3-4f9f-9867-9f2c61f60146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:23.022675 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1383/2737] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:23.030716 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43408a51-fefb-4329-b854-e000662884b5 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:23.031010 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43408a51-fefb-4329-b854-e000662884b5 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:23.031292 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43408a51-fefb-4329-b854-e000662884b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:23.049098 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-43408a51-fefb-4329-b854-e000662884b5 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:23.050288 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-43408a51-fefb-4329-b854-e000662884b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:23.058501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1355/2738] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:23.062622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe52838-8df3-4f9f-9867-9f2c61f60146 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:23.072896 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe52838-8df3-4f9f-9867-9f2c61f60146 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:23.077458 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6cbaa-bfb5-4ed6-be49-2f290878f941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:23.078139 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6cbaa-bfb5-4ed6-be49-2f290878f941 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:23.078424 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6cbaa-bfb5-4ed6-be49-2f290878f941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:23.078703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6cbaa-bfb5-4ed6-be49-2f290878f941 admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:23.079582 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6cbaa-bfb5-4ed6-be49-2f290878f941 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:56:23.079888 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6cbaa-bfb5-4ed6-be49-2f290878f941 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:23.081565 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6cbaa-bfb5-4ed6-be49-2f290878f941 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:23.089177 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:23.095797 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1384/2739] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:56:23.100719 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43408a51-fefb-4329-b854-e000662884b5 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:23.109835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43408a51-fefb-4329-b854-e000662884b5 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:23.115036 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c64f01-24a1-426a-ad46-adbaced8c89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:23.115685 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c64f01-24a1-426a-ad46-adbaced8c89f admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:23.115968 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c64f01-24a1-426a-ad46-adbaced8c89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:23.116245 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c64f01-24a1-426a-ad46-adbaced8c89f admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:23.119134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c64f01-24a1-426a-ad46-adbaced8c89f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:23.119134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c64f01-24a1-426a-ad46-adbaced8c89f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:23.204610 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1356/2740] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:56:23 2022] POST /identity/v3/projects => generated 405 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:56:23.207917 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6cbaa-bfb5-4ed6-be49-2f290878f941 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:23.218566 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6cbaa-bfb5-4ed6-be49-2f290878f941 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:23.275798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77292594-60d3-4a14-8c5b-bbfc5316fb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:23.276502 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77292594-60d3-4a14-8c5b-bbfc5316fb0e admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:23.276813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77292594-60d3-4a14-8c5b-bbfc5316fb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:23.277088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77292594-60d3-4a14-8c5b-bbfc5316fb0e admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:23.277764 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77292594-60d3-4a14-8c5b-bbfc5316fb0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:23.279667 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77292594-60d3-4a14-8c5b-bbfc5316fb0e admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:23.345148 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1385/2741] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:56:23 2022] POST /identity/v3/users => generated 303 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:56:23.349970 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c64f01-24a1-426a-ad46-adbaced8c89f admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:23.360124 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c64f01-24a1-426a-ad46-adbaced8c89f admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:23.364697 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adede1-40d4-4f52-96f4-22aff96f4823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:23.365448 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adede1-40d4-4f52-96f4-22aff96f4823 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:23.365737 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adede1-40d4-4f52-96f4-22aff96f4823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:23.366018 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adede1-40d4-4f52-96f4-22aff96f4823 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:23.366787 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adede1-40d4-4f52-96f4-22aff96f4823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:23.368387 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adede1-40d4-4f52-96f4-22aff96f4823 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:23.378562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1357/2742] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:56:23.383282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77292594-60d3-4a14-8c5b-bbfc5316fb0e admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:23.393270 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77292594-60d3-4a14-8c5b-bbfc5316fb0e admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:23.398613 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86adf2ff-e001-48c8-b342-a650f7165816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:23.399290 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adf2ff-e001-48c8-b342-a650f7165816 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:23.399573 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adf2ff-e001-48c8-b342-a650f7165816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:23.399850 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adf2ff-e001-48c8-b342-a650f7165816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:23.400561 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86adf2ff-e001-48c8-b342-a650f7165816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:23.402158 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86adf2ff-e001-48c8-b342-a650f7165816 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:23.411857 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1386/2743] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:56:23.415836 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adede1-40d4-4f52-96f4-22aff96f4823 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:23.425816 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adede1-40d4-4f52-96f4-22aff96f4823 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:23.431053 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283015c2-6aea-40cc-88e5-854a73c4e48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:23.432172 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283015c2-6aea-40cc-88e5-854a73c4e48f admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:23.432470 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283015c2-6aea-40cc-88e5-854a73c4e48f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:23.432780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283015c2-6aea-40cc-88e5-854a73c4e48f admin admin] PATH_INFO: `/v3/projects/1e19f64e9b1c40f19ada4bdeb9377e0c/users/f375d8302efb43818b6a624a0fe5f05f/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:23.448113 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283015c2-6aea-40cc-88e5-854a73c4e48f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e19f64e9b1c40f19ada4bdeb9377e0c, user_id=f375d8302efb43818b6a624a0fe5f05f, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:23.450909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283015c2-6aea-40cc-88e5-854a73c4e48f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:23.516571 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1358/2744] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:56:23 2022] PUT /identity/v3/projects/1e19f64e9b1c40f19ada4bdeb9377e0c/users/f375d8302efb43818b6a624a0fe5f05f/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:23.527469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d4347-1455-49f9-a8d6-5ab503e58efc None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:23.527766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d4347-1455-49f9-a8d6-5ab503e58efc None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:23.528052 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d4347-1455-49f9-a8d6-5ab503e58efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:23.554125 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-d09d4347-1455-49f9-a8d6-5ab503e58efc None None] MFA Rules not processed for user `f375d8302efb43818b6a624a0fe5f05f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:23.555363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d09d4347-1455-49f9-a8d6-5ab503e58efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:23.619311 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1387/2745] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:23 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:23.623736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283015c2-6aea-40cc-88e5-854a73c4e48f admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:23.684064 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283015c2-6aea-40cc-88e5-854a73c4e48f admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:23.688714 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600a2cf-445c-4b51-91f0-8e492ef84a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:23.689369 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600a2cf-445c-4b51-91f0-8e492ef84a42 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:23.689656 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600a2cf-445c-4b51-91f0-8e492ef84a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:23.689934 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600a2cf-445c-4b51-91f0-8e492ef84a42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:23.690614 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600a2cf-445c-4b51-91f0-8e492ef84a42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:23.692527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600a2cf-445c-4b51-91f0-8e492ef84a42 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:23.793196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1359/2746] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:56:23 2022] POST /identity/v3/projects => generated 405 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:56:23.797129 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d4347-1455-49f9-a8d6-5ab503e58efc None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:23.808598 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d4347-1455-49f9-a8d6-5ab503e58efc None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:23.864465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc2ca0b-076f-4726-8cd2-1cfaba0d73c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:23.865839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2ca0b-076f-4726-8cd2-1cfaba0d73c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:23.866128 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2ca0b-076f-4726-8cd2-1cfaba0d73c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:23.866410 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2ca0b-076f-4726-8cd2-1cfaba0d73c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:23.867110 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc2ca0b-076f-4726-8cd2-1cfaba0d73c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:23.869095 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc2ca0b-076f-4726-8cd2-1cfaba0d73c8 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:23.878710 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:23.975383 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1388/2747] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:56:23 2022] POST /identity/v3/users => generated 309 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:56:23.979048 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600a2cf-445c-4b51-91f0-8e492ef84a42 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:23.990323 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600a2cf-445c-4b51-91f0-8e492ef84a42 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:23.994981 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f657e92-b718-4ed7-8a3d-64dcc99a9bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:23.995645 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f657e92-b718-4ed7-8a3d-64dcc99a9bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:23.995932 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f657e92-b718-4ed7-8a3d-64dcc99a9bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:23.996209 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f657e92-b718-4ed7-8a3d-64dcc99a9bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:23.996968 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f657e92-b718-4ed7-8a3d-64dcc99a9bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:23.998558 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f657e92-b718-4ed7-8a3d-64dcc99a9bb1 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:24.010101 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1360/2748] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:56:24.014220 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc2ca0b-076f-4726-8cd2-1cfaba0d73c8 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:24.024374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc2ca0b-076f-4726-8cd2-1cfaba0d73c8 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:24.028941 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac16aa-99cb-4893-ab3b-921a8fc87984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:24.029864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac16aa-99cb-4893-ab3b-921a8fc87984 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:24.030145 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac16aa-99cb-4893-ab3b-921a8fc87984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:24.030420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac16aa-99cb-4893-ab3b-921a8fc87984 admin admin] PATH_INFO: `/v3/projects/bc0d03b7296f4d4583ed756a7fcadb5e/users/b80e0b10f9744c3db3f5743911e765fe/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:24.045620 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac16aa-99cb-4893-ab3b-921a8fc87984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc0d03b7296f4d4583ed756a7fcadb5e, user_id=b80e0b10f9744c3db3f5743911e765fe, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:24.048191 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac16aa-99cb-4893-ab3b-921a8fc87984 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:24.119282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1389/2749] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:56:24 2022] PUT /identity/v3/projects/bc0d03b7296f4d4583ed756a7fcadb5e/users/b80e0b10f9744c3db3f5743911e765fe/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:56:24.123349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f657e92-b718-4ed7-8a3d-64dcc99a9bb1 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:24.133422 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f657e92-b718-4ed7-8a3d-64dcc99a9bb1 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:24.166783 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:24.189491 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075a030-ebb7-43bc-a349-410f0c8a6582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:24.190158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075a030-ebb7-43bc-a349-410f0c8a6582 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:24.190445 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075a030-ebb7-43bc-a349-410f0c8a6582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:24.190724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075a030-ebb7-43bc-a349-410f0c8a6582 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:24.191450 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075a030-ebb7-43bc-a349-410f0c8a6582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:24.193140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075a030-ebb7-43bc-a349-410f0c8a6582 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:24.203219 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1361/2750] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:56:24.207367 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac16aa-99cb-4893-ab3b-921a8fc87984 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:24.218799 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac16aa-99cb-4893-ab3b-921a8fc87984 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:24.223829 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350ddc6-5e39-4809-93b0-59d49374188d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:24.225268 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350ddc6-5e39-4809-93b0-59d49374188d admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:24.225547 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350ddc6-5e39-4809-93b0-59d49374188d admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:24.225824 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350ddc6-5e39-4809-93b0-59d49374188d admin admin] PATH_INFO: `/v3/projects/bc0d03b7296f4d4583ed756a7fcadb5e/users/b80e0b10f9744c3db3f5743911e765fe/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:24.227795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350ddc6-5e39-4809-93b0-59d49374188d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc0d03b7296f4d4583ed756a7fcadb5e, user_id=b80e0b10f9744c3db3f5743911e765fe, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:24.230513 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350ddc6-5e39-4809-93b0-59d49374188d admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:24.240982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1390/2751] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:56:24 2022] PUT /identity/v3/projects/bc0d03b7296f4d4583ed756a7fcadb5e/users/b80e0b10f9744c3db3f5743911e765fe/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:24.250878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4a456-4269-4466-be61-a97c4fb40079 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:24.251172 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4a456-4269-4466-be61-a97c4fb40079 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:24.251454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4a456-4269-4466-be61-a97c4fb40079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:24.279238 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-51b4a456-4269-4466-be61-a97c4fb40079 None None] MFA Rules not processed for user `b80e0b10f9744c3db3f5743911e765fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:24.280453 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-51b4a456-4269-4466-be61-a97c4fb40079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:24.347075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1362/2752] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:24 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:24.358220 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350ddc6-5e39-4809-93b0-59d49374188d admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:24.360046 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-7350ddc6-5e39-4809-93b0-59d49374188d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:24.418120 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350ddc6-5e39-4809-93b0-59d49374188d admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:24.421957 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ffb27-e68a-47af-b057-06093d0bdbb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:24.422618 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ffb27-e68a-47af-b057-06093d0bdbb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:24.422889 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ffb27-e68a-47af-b057-06093d0bdbb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:24.423166 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ffb27-e68a-47af-b057-06093d0bdbb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:24.435148 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ffb27-e68a-47af-b057-06093d0bdbb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:24.437401 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ffb27-e68a-47af-b057-06093d0bdbb5 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:24.447032 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1391/2753] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:56:24 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:56:24.574264 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4a456-4269-4466-be61-a97c4fb40079 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:24.638708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4a456-4269-4466-be61-a97c4fb40079 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:24.643030 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f762b2b-5311-414c-9ed8-e236198ecbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:24.643898 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f762b2b-5311-414c-9ed8-e236198ecbe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:24.644176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f762b2b-5311-414c-9ed8-e236198ecbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:24.644474 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f762b2b-5311-414c-9ed8-e236198ecbe0 admin admin] PATH_INFO: `/v3/users/f375d8302efb43818b6a624a0fe5f05f` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:24.645666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f762b2b-5311-414c-9ed8-e236198ecbe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f375d8302efb43818b6a624a0fe5f05f)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:24.648247 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f762b2b-5311-414c-9ed8-e236198ecbe0 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:24.844158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9f762b2b-5311-414c-9ed8-e236198ecbe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f375d8302efb43818b6a624a0fe5f05f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:24.856822 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:24.858705 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9f762b2b-5311-414c-9ed8-e236198ecbe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f375d8302efb43818b6a624a0fe5f05f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:24.922442 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9f762b2b-5311-414c-9ed8-e236198ecbe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f375d8302efb43818b6a624a0fe5f05f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:24.922989 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9f762b2b-5311-414c-9ed8-e236198ecbe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f375d8302efb43818b6a624a0fe5f05f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:24.923591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-9f762b2b-5311-414c-9ed8-e236198ecbe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f375d8302efb43818b6a624a0fe5f05f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:24.936620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1363/2754] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:56:24 2022] DELETE /identity/v3/users/f375d8302efb43818b6a624a0fe5f05f => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:24.941251 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16812d-e59c-4d49-adb7-8f5e7e78df46 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:24.941616 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16812d-e59c-4d49-adb7-8f5e7e78df46 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:24.941897 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16812d-e59c-4d49-adb7-8f5e7e78df46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:24.960264 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-3e16812d-e59c-4d49-adb7-8f5e7e78df46 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:24.961532 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3e16812d-e59c-4d49-adb7-8f5e7e78df46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:25.020672 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1392/2755] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:25.030425 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f762b2b-5311-414c-9ed8-e236198ecbe0 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:25.032379 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-9f762b2b-5311-414c-9ed8-e236198ecbe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:25.089313 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f762b2b-5311-414c-9ed8-e236198ecbe0 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:25.092993 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76013b99-497c-441d-a7ef-67856a763bf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:25.093640 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76013b99-497c-441d-a7ef-67856a763bf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:25.093924 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76013b99-497c-441d-a7ef-67856a763bf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:25.094203 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76013b99-497c-441d-a7ef-67856a763bf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:25.103693 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76013b99-497c-441d-a7ef-67856a763bf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:25.105237 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76013b99-497c-441d-a7ef-67856a763bf3 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:25.115252 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1364/2756] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:56:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:56:25.747429 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16812d-e59c-4d49-adb7-8f5e7e78df46 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:25.748968 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-3e16812d-e59c-4d49-adb7-8f5e7e78df46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:25.761789 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16812d-e59c-4d49-adb7-8f5e7e78df46 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:25.767000 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e7826-6238-412e-a657-e5431717a700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:25.767903 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e7826-6238-412e-a657-e5431717a700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:25.768187 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e7826-6238-412e-a657-e5431717a700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:25.768908 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e7826-6238-412e-a657-e5431717a700 admin admin] PATH_INFO: `/v3/projects/1e19f64e9b1c40f19ada4bdeb9377e0c` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:25.769784 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e7826-6238-412e-a657-e5431717a700 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e19f64e9b1c40f19ada4bdeb9377e0c)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:25.774967 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e7826-6238-412e-a657-e5431717a700 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:25.894305 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-5f4e7826-6238-412e-a657-e5431717a700 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e19f64e9b1c40f19ada4bdeb9377e0c', 'request_id': 'req-5f4e7826-6238-412e-a657-e5431717a700'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:25.900485 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-5f4e7826-6238-412e-a657-e5431717a700 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e19f64e9b1c40f19ada4bdeb9377e0c', 'request_id': 'req-5f4e7826-6238-412e-a657-e5431717a700'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:25.902534 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:25.906306 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-5f4e7826-6238-412e-a657-e5431717a700 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e19f64e9b1c40f19ada4bdeb9377e0c', 'request_id': 'req-5f4e7826-6238-412e-a657-e5431717a700'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:25.992680 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-5f4e7826-6238-412e-a657-e5431717a700 admin admin] The token cache is being invalidate because project 1e19f64e9b1c40f19ada4bdeb9377e0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:56:25.992970 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-5f4e7826-6238-412e-a657-e5431717a700 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:25.993614 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-5f4e7826-6238-412e-a657-e5431717a700 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:25.995160 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1393/2757] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:56:25 2022] DELETE /identity/v3/projects/1e19f64e9b1c40f19ada4bdeb9377e0c => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:25.999130 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76013b99-497c-441d-a7ef-67856a763bf3 service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:26.009728 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76013b99-497c-441d-a7ef-67856a763bf3 service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:26.066638 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88ba7b-ab65-4ef6-9990-52ae7b442f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:26.067503 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88ba7b-ab65-4ef6-9990-52ae7b442f34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:26.067811 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88ba7b-ab65-4ef6-9990-52ae7b442f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:26.068094 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88ba7b-ab65-4ef6-9990-52ae7b442f34 admin admin] PATH_INFO: `/v3/users/b80e0b10f9744c3db3f5743911e765fe` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:26.069263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88ba7b-ab65-4ef6-9990-52ae7b442f34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b80e0b10f9744c3db3f5743911e765fe)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:26.071307 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88ba7b-ab65-4ef6-9990-52ae7b442f34 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:26.239039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fb88ba7b-ab65-4ef6-9990-52ae7b442f34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b80e0b10f9744c3db3f5743911e765fe'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:26.251468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fb88ba7b-ab65-4ef6-9990-52ae7b442f34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b80e0b10f9744c3db3f5743911e765fe'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:26.316727 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fb88ba7b-ab65-4ef6-9990-52ae7b442f34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b80e0b10f9744c3db3f5743911e765fe'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:26.317340 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fb88ba7b-ab65-4ef6-9990-52ae7b442f34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b80e0b10f9744c3db3f5743911e765fe'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:26.317997 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-fb88ba7b-ab65-4ef6-9990-52ae7b442f34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b80e0b10f9744c3db3f5743911e765fe'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:26.326867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:26.331095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1365/2758] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:56:25 2022] DELETE /identity/v3/users/b80e0b10f9744c3db3f5743911e765fe => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:26.749363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e7826-6238-412e-a657-e5431717a700 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:26.750826 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-5f4e7826-6238-412e-a657-e5431717a700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:26.766186 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e7826-6238-412e-a657-e5431717a700 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:26.827516 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976c36a-7a8b-4211-aa50-dfb4ccb70b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:26.828389 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976c36a-7a8b-4211-aa50-dfb4ccb70b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:26.828771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976c36a-7a8b-4211-aa50-dfb4ccb70b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:26.828984 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976c36a-7a8b-4211-aa50-dfb4ccb70b3d admin admin] PATH_INFO: `/v3/projects/bc0d03b7296f4d4583ed756a7fcadb5e` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:26.830288 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e976c36a-7a8b-4211-aa50-dfb4ccb70b3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc0d03b7296f4d4583ed756a7fcadb5e)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:26.832312 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e976c36a-7a8b-4211-aa50-dfb4ccb70b3d admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:26.947539 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e976c36a-7a8b-4211-aa50-dfb4ccb70b3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc0d03b7296f4d4583ed756a7fcadb5e', 'request_id': 'req-e976c36a-7a8b-4211-aa50-dfb4ccb70b3d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:26.953632 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e976c36a-7a8b-4211-aa50-dfb4ccb70b3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc0d03b7296f4d4583ed756a7fcadb5e', 'request_id': 'req-e976c36a-7a8b-4211-aa50-dfb4ccb70b3d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:26.959652 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e976c36a-7a8b-4211-aa50-dfb4ccb70b3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc0d03b7296f4d4583ed756a7fcadb5e', 'request_id': 'req-e976c36a-7a8b-4211-aa50-dfb4ccb70b3d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:27.047303 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e976c36a-7a8b-4211-aa50-dfb4ccb70b3d admin admin] The token cache is being invalidate because project bc0d03b7296f4d4583ed756a7fcadb5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:56:27.047525 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e976c36a-7a8b-4211-aa50-dfb4ccb70b3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:27.048137 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e976c36a-7a8b-4211-aa50-dfb4ccb70b3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:27.049734 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1394/2759] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:56:26 2022] DELETE /identity/v3/projects/bc0d03b7296f4d4583ed756a7fcadb5e => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:27.054881 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4792fa-8521-4605-8d05-dffb0887dfe2 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:27.055374 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4792fa-8521-4605-8d05-dffb0887dfe2 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:27.055507 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4792fa-8521-4605-8d05-dffb0887dfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:27.074079 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-4c4792fa-8521-4605-8d05-dffb0887dfe2 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:27.075345 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-4c4792fa-8521-4605-8d05-dffb0887dfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:27.133660 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1366/2760] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:27.141487 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66ec72-1c8f-4bf9-a241-fb0edefaa299 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:27.141777 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66ec72-1c8f-4bf9-a241-fb0edefaa299 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:27.142056 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66ec72-1c8f-4bf9-a241-fb0edefaa299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:27.162196 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-9a66ec72-1c8f-4bf9-a241-fb0edefaa299 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:27.163387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-9a66ec72-1c8f-4bf9-a241-fb0edefaa299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:27.171173 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1395/2761] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:27.175224 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4792fa-8521-4605-8d05-dffb0887dfe2 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:27.185493 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4792fa-8521-4605-8d05-dffb0887dfe2 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:27.190236 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4cc0f-1fa2-474d-9112-eaead483c871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:27.190889 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4cc0f-1fa2-474d-9112-eaead483c871 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:27.191172 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4cc0f-1fa2-474d-9112-eaead483c871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:27.191450 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4cc0f-1fa2-474d-9112-eaead483c871 admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:27.192173 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4cc0f-1fa2-474d-9112-eaead483c871 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:56:27.193129 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4cc0f-1fa2-474d-9112-eaead483c871 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:27.194803 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4cc0f-1fa2-474d-9112-eaead483c871 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:27.206427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1367/2762] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:56:27.210860 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66ec72-1c8f-4bf9-a241-fb0edefaa299 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:27.220043 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66ec72-1c8f-4bf9-a241-fb0edefaa299 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:27.225061 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef44ed-c3de-4b58-878a-261e29fbfdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:27.225702 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef44ed-c3de-4b58-878a-261e29fbfdf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:27.225985 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef44ed-c3de-4b58-878a-261e29fbfdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:27.226257 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef44ed-c3de-4b58-878a-261e29fbfdf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:27.226941 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef44ed-c3de-4b58-878a-261e29fbfdf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:27.228849 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef44ed-c3de-4b58-878a-261e29fbfdf5 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:27.315242 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1396/2763] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:56:27 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:56:27.316457 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4cc0f-1fa2-474d-9112-eaead483c871 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:27.327056 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4cc0f-1fa2-474d-9112-eaead483c871 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:27.382581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f4ce0-911b-4163-98e1-2668eab66537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:27.383239 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f4ce0-911b-4163-98e1-2668eab66537 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:27.383521 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f4ce0-911b-4163-98e1-2668eab66537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:27.383795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f4ce0-911b-4163-98e1-2668eab66537 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:27.384479 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f4ce0-911b-4163-98e1-2668eab66537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:27.386417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f4ce0-911b-4163-98e1-2668eab66537 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:27.499200 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1368/2764] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:56:27 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:56:27.503319 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef44ed-c3de-4b58-878a-261e29fbfdf5 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:27.513997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef44ed-c3de-4b58-878a-261e29fbfdf5 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:27.518692 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3b2be-c5c5-48a5-a5e1-fa0569b834f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:27.519340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3b2be-c5c5-48a5-a5e1-fa0569b834f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:27.519622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3b2be-c5c5-48a5-a5e1-fa0569b834f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:27.519895 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3b2be-c5c5-48a5-a5e1-fa0569b834f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:27.520671 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c3b2be-c5c5-48a5-a5e1-fa0569b834f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:27.522249 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c3b2be-c5c5-48a5-a5e1-fa0569b834f3 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:27.532451 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1397/2765] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:56:27.536837 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f4ce0-911b-4163-98e1-2668eab66537 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:27.546874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f4ce0-911b-4163-98e1-2668eab66537 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:27.551439 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf3759-1217-423e-8f43-352397a5e98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:27.553167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf3759-1217-423e-8f43-352397a5e98a admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:27.553461 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf3759-1217-423e-8f43-352397a5e98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:27.553744 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf3759-1217-423e-8f43-352397a5e98a admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:27.554469 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf3759-1217-423e-8f43-352397a5e98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:27.556087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf3759-1217-423e-8f43-352397a5e98a admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:27.565989 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1369/2766] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:56:27.570422 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3b2be-c5c5-48a5-a5e1-fa0569b834f3 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:27.580133 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3b2be-c5c5-48a5-a5e1-fa0569b834f3 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:27.585317 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93701c-3570-4b04-8b82-8ebbaa987e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:27.586174 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93701c-3570-4b04-8b82-8ebbaa987e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:27.586459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93701c-3570-4b04-8b82-8ebbaa987e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:27.586734 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93701c-3570-4b04-8b82-8ebbaa987e64 admin admin] PATH_INFO: `/v3/projects/77258f841f6d4b3ab9f761e56c78f002/users/45b65cee84384b818203374bc86ac5ec/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:27.600949 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93701c-3570-4b04-8b82-8ebbaa987e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77258f841f6d4b3ab9f761e56c78f002, user_id=45b65cee84384b818203374bc86ac5ec, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:27.603580 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93701c-3570-4b04-8b82-8ebbaa987e64 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:27.701378 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1398/2767] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:56:27 2022] PUT /identity/v3/projects/77258f841f6d4b3ab9f761e56c78f002/users/45b65cee84384b818203374bc86ac5ec/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:56:27.713391 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4c106-2231-48c7-ba5f-aa5e86947a7d None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:27.713689 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4c106-2231-48c7-ba5f-aa5e86947a7d None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:27.713974 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4c106-2231-48c7-ba5f-aa5e86947a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:27.741680 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-53c4c106-2231-48c7-ba5f-aa5e86947a7d None None] MFA Rules not processed for user `45b65cee84384b818203374bc86ac5ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:27.742897 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-53c4c106-2231-48c7-ba5f-aa5e86947a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:27.797277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:27.805866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1370/2768] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:27.810332 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93701c-3570-4b04-8b82-8ebbaa987e64 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:27.823627 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:27.873385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93701c-3570-4b04-8b82-8ebbaa987e64 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:27.878060 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc55326-0797-4d5f-a620-461b1509ecaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:27.878719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc55326-0797-4d5f-a620-461b1509ecaa admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:27.879007 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc55326-0797-4d5f-a620-461b1509ecaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:27.879286 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc55326-0797-4d5f-a620-461b1509ecaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:27.880548 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc55326-0797-4d5f-a620-461b1509ecaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:27.882621 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc55326-0797-4d5f-a620-461b1509ecaa admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:28.011926 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1399/2769] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:56:27 2022] POST /identity/v3/projects => generated 379 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:56:28.016132 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4c106-2231-48c7-ba5f-aa5e86947a7d None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:28.026891 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4c106-2231-48c7-ba5f-aa5e86947a7d None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:28.083069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacca34a-a089-492a-866c-0573df0ab2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:28.083793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacca34a-a089-492a-866c-0573df0ab2fc admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:28.084046 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacca34a-a089-492a-866c-0573df0ab2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:28.084224 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacca34a-a089-492a-866c-0573df0ab2fc admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:28.085233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacca34a-a089-492a-866c-0573df0ab2fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:28.086894 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacca34a-a089-492a-866c-0573df0ab2fc admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:28.158413 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1371/2770] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:56:28 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:56:28.162689 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc55326-0797-4d5f-a620-461b1509ecaa admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:28.173011 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc55326-0797-4d5f-a620-461b1509ecaa admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:28.178490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9296f-3cc4-4bf0-9f5b-681ba2b56c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:28.179152 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9296f-3cc4-4bf0-9f5b-681ba2b56c5b admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:28.179444 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9296f-3cc4-4bf0-9f5b-681ba2b56c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:28.179727 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9296f-3cc4-4bf0-9f5b-681ba2b56c5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:28.180460 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9296f-3cc4-4bf0-9f5b-681ba2b56c5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:28.182094 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9296f-3cc4-4bf0-9f5b-681ba2b56c5b admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:28.193399 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1400/2771] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:56:28.197167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacca34a-a089-492a-866c-0573df0ab2fc admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:28.207035 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacca34a-a089-492a-866c-0573df0ab2fc admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:28.211929 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db2461-849d-4ac3-ad53-7165323588c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:28.212620 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db2461-849d-4ac3-ad53-7165323588c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:28.212909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db2461-849d-4ac3-ad53-7165323588c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:28.213185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db2461-849d-4ac3-ad53-7165323588c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:28.213911 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47db2461-849d-4ac3-ad53-7165323588c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:28.215565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47db2461-849d-4ac3-ad53-7165323588c8 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:28.225865 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1372/2772] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:56:28.230176 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9296f-3cc4-4bf0-9f5b-681ba2b56c5b admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:28.240929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9296f-3cc4-4bf0-9f5b-681ba2b56c5b admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:28.245459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fc8f0-54af-4657-a9eb-921d915d7c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:28.249495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fc8f0-54af-4657-a9eb-921d915d7c39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:28.249495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fc8f0-54af-4657-a9eb-921d915d7c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:28.249495 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fc8f0-54af-4657-a9eb-921d915d7c39 admin admin] PATH_INFO: `/v3/projects/a4646d44d71b4cf685a5a6f2723b5a6a/users/edc466c195e54682940cc83b5e438df3/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:28.261994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260fc8f0-54af-4657-a9eb-921d915d7c39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4646d44d71b4cf685a5a6f2723b5a6a, user_id=edc466c195e54682940cc83b5e438df3, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:28.264659 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260fc8f0-54af-4657-a9eb-921d915d7c39 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:28.328272 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1401/2773] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:56:28 2022] PUT /identity/v3/projects/a4646d44d71b4cf685a5a6f2723b5a6a/users/edc466c195e54682940cc83b5e438df3/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:28.338392 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943779c-6c06-484e-9b5f-f0a66016c088 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:28.338631 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943779c-6c06-484e-9b5f-f0a66016c088 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:28.338907 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943779c-6c06-484e-9b5f-f0a66016c088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:28.366078 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-2943779c-6c06-484e-9b5f-f0a66016c088 None None] MFA Rules not processed for user `edc466c195e54682940cc83b5e438df3`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:28.367272 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-2943779c-6c06-484e-9b5f-f0a66016c088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:28.429235 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1373/2774] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:28.433654 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fc8f0-54af-4657-a9eb-921d915d7c39 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:28.494263 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fc8f0-54af-4657-a9eb-921d915d7c39 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:28.498864 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a087dc-d89c-44bc-a00d-5cf52d31ec07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:28.499516 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a087dc-d89c-44bc-a00d-5cf52d31ec07 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:28.499796 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a087dc-d89c-44bc-a00d-5cf52d31ec07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:28.500071 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a087dc-d89c-44bc-a00d-5cf52d31ec07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:28.500783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a087dc-d89c-44bc-a00d-5cf52d31ec07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:28.502672 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a087dc-d89c-44bc-a00d-5cf52d31ec07 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:28.628857 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1402/2775] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:56:28 2022] POST /identity/v3/projects => generated 375 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:56:28.632872 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943779c-6c06-484e-9b5f-f0a66016c088 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:28.643393 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943779c-6c06-484e-9b5f-f0a66016c088 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:28.698454 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e1fe6-d4b5-469d-8e0d-d49f9cdf0dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:28.699148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e1fe6-d4b5-469d-8e0d-d49f9cdf0dbc admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:28.699430 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e1fe6-d4b5-469d-8e0d-d49f9cdf0dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:28.699710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e1fe6-d4b5-469d-8e0d-d49f9cdf0dbc admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:28.700402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e1fe6-d4b5-469d-8e0d-d49f9cdf0dbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:28.702349 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e1fe6-d4b5-469d-8e0d-d49f9cdf0dbc admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:28.711807 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:28.804560 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1374/2776] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:56:28 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:56:28.808516 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a087dc-d89c-44bc-a00d-5cf52d31ec07 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:28.819265 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a087dc-d89c-44bc-a00d-5cf52d31ec07 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:28.823948 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef492a3a-4f02-43f9-87ef-54cc5781c837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:28.824630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef492a3a-4f02-43f9-87ef-54cc5781c837 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:28.824920 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef492a3a-4f02-43f9-87ef-54cc5781c837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:28.825202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef492a3a-4f02-43f9-87ef-54cc5781c837 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:28.826042 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef492a3a-4f02-43f9-87ef-54cc5781c837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:28.827634 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef492a3a-4f02-43f9-87ef-54cc5781c837 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:28.838842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1403/2777] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:56:28.842918 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e1fe6-d4b5-469d-8e0d-d49f9cdf0dbc admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:28.853739 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e1fe6-d4b5-469d-8e0d-d49f9cdf0dbc admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:28.858260 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b01abc-1078-4e7e-9af5-08113be9228b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:28.859134 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b01abc-1078-4e7e-9af5-08113be9228b admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:28.859418 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b01abc-1078-4e7e-9af5-08113be9228b admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:28.859696 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b01abc-1078-4e7e-9af5-08113be9228b admin admin] PATH_INFO: `/v3/projects/f6194539d3fb44549b899ca399f342a4/users/c0a97c3801a24590ab0f9d48a863e809/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:28.874279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b01abc-1078-4e7e-9af5-08113be9228b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6194539d3fb44549b899ca399f342a4, user_id=c0a97c3801a24590ab0f9d48a863e809, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:28.876950 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b01abc-1078-4e7e-9af5-08113be9228b admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:28.941582 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1375/2778] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:56:28 2022] PUT /identity/v3/projects/f6194539d3fb44549b899ca399f342a4/users/c0a97c3801a24590ab0f9d48a863e809/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:28.945609 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef492a3a-4f02-43f9-87ef-54cc5781c837 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:28.955649 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef492a3a-4f02-43f9-87ef-54cc5781c837 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:28.988945 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:29.012763 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dd868-cf31-4943-a4c4-4909a6edc284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:29.013431 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dd868-cf31-4943-a4c4-4909a6edc284 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:29.013719 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dd868-cf31-4943-a4c4-4909a6edc284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:29.014115 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dd868-cf31-4943-a4c4-4909a6edc284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:29.014842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35dd868-cf31-4943-a4c4-4909a6edc284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:29.016426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35dd868-cf31-4943-a4c4-4909a6edc284 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:29.027281 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1404/2779] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:56:29.031467 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b01abc-1078-4e7e-9af5-08113be9228b admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:29.041832 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b01abc-1078-4e7e-9af5-08113be9228b admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:29.046533 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e7f13-6f38-40a3-bb59-0322b49df73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:29.047405 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e7f13-6f38-40a3-bb59-0322b49df73a admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:29.047689 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e7f13-6f38-40a3-bb59-0322b49df73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:29.047966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e7f13-6f38-40a3-bb59-0322b49df73a admin admin] PATH_INFO: `/v3/projects/f6194539d3fb44549b899ca399f342a4/users/c0a97c3801a24590ab0f9d48a863e809/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:29.050072 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e7f13-6f38-40a3-bb59-0322b49df73a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6194539d3fb44549b899ca399f342a4, user_id=c0a97c3801a24590ab0f9d48a863e809, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:29.052881 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e7f13-6f38-40a3-bb59-0322b49df73a admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:29.062883 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1376/2780] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:56:29 2022] PUT /identity/v3/projects/f6194539d3fb44549b899ca399f342a4/users/c0a97c3801a24590ab0f9d48a863e809/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:29.073315 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba0bbe-21b5-418a-89da-bba2ea2eae8e None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:29.073638 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba0bbe-21b5-418a-89da-bba2ea2eae8e None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:29.073928 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba0bbe-21b5-418a-89da-bba2ea2eae8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:29.100972 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-2eba0bbe-21b5-418a-89da-bba2ea2eae8e None None] MFA Rules not processed for user `c0a97c3801a24590ab0f9d48a863e809`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:29.102188 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-2eba0bbe-21b5-418a-89da-bba2ea2eae8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:29.168644 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1405/2781] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:29.179622 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e7f13-6f38-40a3-bb59-0322b49df73a admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:29.181405 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-549e7f13-6f38-40a3-bb59-0322b49df73a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:29.237872 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e7f13-6f38-40a3-bb59-0322b49df73a admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:29.241829 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1cd178-4f19-44ab-bc6d-94126fe2d2f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:29.242450 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1cd178-4f19-44ab-bc6d-94126fe2d2f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:29.242730 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1cd178-4f19-44ab-bc6d-94126fe2d2f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:29.243008 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1cd178-4f19-44ab-bc6d-94126fe2d2f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:29.251871 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1cd178-4f19-44ab-bc6d-94126fe2d2f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:29.253330 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1cd178-4f19-44ab-bc6d-94126fe2d2f6 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:29.263069 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1377/2782] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:56:29 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:56:29.366873 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba0bbe-21b5-418a-89da-bba2ea2eae8e None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:29.368289 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-2eba0bbe-21b5-418a-89da-bba2ea2eae8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:29.381611 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba0bbe-21b5-418a-89da-bba2ea2eae8e None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:29.385250 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d71f44-4632-4dd9-af03-4a6fa5e5b112 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:29.385896 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d71f44-4632-4dd9-af03-4a6fa5e5b112 service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:29.386184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d71f44-4632-4dd9-af03-4a6fa5e5b112 service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:29.386465 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d71f44-4632-4dd9-af03-4a6fa5e5b112 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:29.444450 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d71f44-4632-4dd9-af03-4a6fa5e5b112 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:29.445913 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d71f44-4632-4dd9-af03-4a6fa5e5b112 service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:29.454839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1406/2783] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:56:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:56:33.338856 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1cd178-4f19-44ab-bc6d-94126fe2d2f6 service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:33.399935 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1cd178-4f19-44ab-bc6d-94126fe2d2f6 service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:33.404386 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6163a8-631e-45ea-a591-ab142628a1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:33.405276 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6163a8-631e-45ea-a591-ab142628a1ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:33.405559 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6163a8-631e-45ea-a591-ab142628a1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:33.405835 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6163a8-631e-45ea-a591-ab142628a1ed admin admin] PATH_INFO: `/v3/users/45b65cee84384b818203374bc86ac5ec` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:33.407001 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6163a8-631e-45ea-a591-ab142628a1ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=45b65cee84384b818203374bc86ac5ec)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:33.408941 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6163a8-631e-45ea-a591-ab142628a1ed admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:33.543031 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-cd6163a8-631e-45ea-a591-ab142628a1ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45b65cee84384b818203374bc86ac5ec'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:33.553279 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:33.555111 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-cd6163a8-631e-45ea-a591-ab142628a1ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45b65cee84384b818203374bc86ac5ec'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:33.604256 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-cd6163a8-631e-45ea-a591-ab142628a1ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45b65cee84384b818203374bc86ac5ec'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:33.604894 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-cd6163a8-631e-45ea-a591-ab142628a1ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45b65cee84384b818203374bc86ac5ec'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:33.605501 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-cd6163a8-631e-45ea-a591-ab142628a1ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45b65cee84384b818203374bc86ac5ec'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:33.618661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1378/2784] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:56:33 2022] DELETE /identity/v3/users/45b65cee84384b818203374bc86ac5ec => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:33.623200 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24331358-e7ec-43e5-a263-075667145cb5 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:33.623682 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24331358-e7ec-43e5-a263-075667145cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:33.623786 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24331358-e7ec-43e5-a263-075667145cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:33.641379 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-24331358-e7ec-43e5-a263-075667145cb5 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:33.642591 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-24331358-e7ec-43e5-a263-075667145cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:33.690443 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=100 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:33.702232 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1407/2785] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:33.712378 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6163a8-631e-45ea-a591-ab142628a1ed admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:33.713867 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-cd6163a8-631e-45ea-a591-ab142628a1ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:33.770255 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6163a8-631e-45ea-a591-ab142628a1ed admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:33.774864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06255c3-3017-4023-9bec-4d4e28cde03e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:33.775515 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06255c3-3017-4023-9bec-4d4e28cde03e service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:33.775805 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06255c3-3017-4023-9bec-4d4e28cde03e service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:33.776087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06255c3-3017-4023-9bec-4d4e28cde03e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:33.785184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06255c3-3017-4023-9bec-4d4e28cde03e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:33.786779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06255c3-3017-4023-9bec-4d4e28cde03e service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:33.796331 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1379/2786] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:56:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:56:34.261796 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24331358-e7ec-43e5-a263-075667145cb5 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:34.263261 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-24331358-e7ec-43e5-a263-075667145cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:34.282232 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24331358-e7ec-43e5-a263-075667145cb5 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:34.288933 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61d7ae-a3f8-4cd0-ac50-e5855a88eb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:34.289726 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61d7ae-a3f8-4cd0-ac50-e5855a88eb43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:34.290011 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61d7ae-a3f8-4cd0-ac50-e5855a88eb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:34.290819 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61d7ae-a3f8-4cd0-ac50-e5855a88eb43 admin admin] PATH_INFO: `/v3/projects/77258f841f6d4b3ab9f761e56c78f002` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:34.291792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61d7ae-a3f8-4cd0-ac50-e5855a88eb43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77258f841f6d4b3ab9f761e56c78f002)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:34.293775 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61d7ae-a3f8-4cd0-ac50-e5855a88eb43 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:34.433065 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-db61d7ae-a3f8-4cd0-ac50-e5855a88eb43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77258f841f6d4b3ab9f761e56c78f002', 'request_id': 'req-db61d7ae-a3f8-4cd0-ac50-e5855a88eb43'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:34.439217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-db61d7ae-a3f8-4cd0-ac50-e5855a88eb43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77258f841f6d4b3ab9f761e56c78f002', 'request_id': 'req-db61d7ae-a3f8-4cd0-ac50-e5855a88eb43'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:34.444734 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-db61d7ae-a3f8-4cd0-ac50-e5855a88eb43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77258f841f6d4b3ab9f761e56c78f002', 'request_id': 'req-db61d7ae-a3f8-4cd0-ac50-e5855a88eb43'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:34.546202 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-db61d7ae-a3f8-4cd0-ac50-e5855a88eb43 admin admin] The token cache is being invalidate because project 77258f841f6d4b3ab9f761e56c78f002 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:56:34.546490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-db61d7ae-a3f8-4cd0-ac50-e5855a88eb43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:34.547111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-db61d7ae-a3f8-4cd0-ac50-e5855a88eb43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:34.548669 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1408/2787] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:56:34 2022] DELETE /identity/v3/projects/77258f841f6d4b3ab9f761e56c78f002 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:34.553378 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06255c3-3017-4023-9bec-4d4e28cde03e service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:34.563056 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06255c3-3017-4023-9bec-4d4e28cde03e service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:34.620595 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7d723-8db5-4dee-8167-0fa4c936af4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:34.621489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7d723-8db5-4dee-8167-0fa4c936af4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:34.621775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7d723-8db5-4dee-8167-0fa4c936af4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:34.622055 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7d723-8db5-4dee-8167-0fa4c936af4e admin admin] PATH_INFO: `/v3/users/edc466c195e54682940cc83b5e438df3` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:34.623232 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a7d723-8db5-4dee-8167-0fa4c936af4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=edc466c195e54682940cc83b5e438df3)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:34.625184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a7d723-8db5-4dee-8167-0fa4c936af4e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:34.805814 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a2a7d723-8db5-4dee-8167-0fa4c936af4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edc466c195e54682940cc83b5e438df3'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:34.817805 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a2a7d723-8db5-4dee-8167-0fa4c936af4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edc466c195e54682940cc83b5e438df3'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:34.893438 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a2a7d723-8db5-4dee-8167-0fa4c936af4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edc466c195e54682940cc83b5e438df3'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:34.894060 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a2a7d723-8db5-4dee-8167-0fa4c936af4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edc466c195e54682940cc83b5e438df3'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:34.894666 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-a2a7d723-8db5-4dee-8167-0fa4c936af4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edc466c195e54682940cc83b5e438df3'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:34.903872 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:34.908016 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1380/2788] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:56:34 2022] DELETE /identity/v3/users/edc466c195e54682940cc83b5e438df3 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:35.331651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61d7ae-a3f8-4cd0-ac50-e5855a88eb43 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:35.333698 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-db61d7ae-a3f8-4cd0-ac50-e5855a88eb43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:35.350235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61d7ae-a3f8-4cd0-ac50-e5855a88eb43 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:35.406307 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2eec2-0e8b-4681-97d3-0bc1958241bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:35.407171 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2eec2-0e8b-4681-97d3-0bc1958241bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:35.407454 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2eec2-0e8b-4681-97d3-0bc1958241bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:35.407729 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2eec2-0e8b-4681-97d3-0bc1958241bc admin admin] PATH_INFO: `/v3/projects/a4646d44d71b4cf685a5a6f2723b5a6a` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:35.408938 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd2eec2-0e8b-4681-97d3-0bc1958241bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4646d44d71b4cf685a5a6f2723b5a6a)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:35.411393 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd2eec2-0e8b-4681-97d3-0bc1958241bc admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:35.547340 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-7cd2eec2-0e8b-4681-97d3-0bc1958241bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4646d44d71b4cf685a5a6f2723b5a6a', 'request_id': 'req-7cd2eec2-0e8b-4681-97d3-0bc1958241bc'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:35.553779 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-7cd2eec2-0e8b-4681-97d3-0bc1958241bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4646d44d71b4cf685a5a6f2723b5a6a', 'request_id': 'req-7cd2eec2-0e8b-4681-97d3-0bc1958241bc'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:35.559242 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-7cd2eec2-0e8b-4681-97d3-0bc1958241bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4646d44d71b4cf685a5a6f2723b5a6a', 'request_id': 'req-7cd2eec2-0e8b-4681-97d3-0bc1958241bc'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:35.648085 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-7cd2eec2-0e8b-4681-97d3-0bc1958241bc admin admin] The token cache is being invalidate because project a4646d44d71b4cf685a5a6f2723b5a6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:56:35.648295 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-7cd2eec2-0e8b-4681-97d3-0bc1958241bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:35.648915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-7cd2eec2-0e8b-4681-97d3-0bc1958241bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:35.650456 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1409/2789] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:56:35 2022] DELETE /identity/v3/projects/a4646d44d71b4cf685a5a6f2723b5a6a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:35.654658 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7d723-8db5-4dee-8167-0fa4c936af4e admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:35.656063 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-a2a7d723-8db5-4dee-8167-0fa4c936af4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:35.669511 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7d723-8db5-4dee-8167-0fa4c936af4e admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:35.727310 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca06f32-cd13-4937-8a1d-6617f84974ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:35.728185 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca06f32-cd13-4937-8a1d-6617f84974ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:35.728472 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca06f32-cd13-4937-8a1d-6617f84974ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:35.728791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca06f32-cd13-4937-8a1d-6617f84974ee admin admin] PATH_INFO: `/v3/users/c0a97c3801a24590ab0f9d48a863e809` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:35.729981 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca06f32-cd13-4937-8a1d-6617f84974ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0a97c3801a24590ab0f9d48a863e809)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:35.732012 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca06f32-cd13-4937-8a1d-6617f84974ee admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:35.925866 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7ca06f32-cd13-4937-8a1d-6617f84974ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0a97c3801a24590ab0f9d48a863e809'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:35.937593 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7ca06f32-cd13-4937-8a1d-6617f84974ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0a97c3801a24590ab0f9d48a863e809'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:36.021704 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7ca06f32-cd13-4937-8a1d-6617f84974ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0a97c3801a24590ab0f9d48a863e809'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:36.022328 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7ca06f32-cd13-4937-8a1d-6617f84974ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0a97c3801a24590ab0f9d48a863e809'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:36.022936 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7ca06f32-cd13-4937-8a1d-6617f84974ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0a97c3801a24590ab0f9d48a863e809'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:36.035489 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1381/2790] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:56:35 2022] DELETE /identity/v3/users/c0a97c3801a24590ab0f9d48a863e809 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:56:36.427189 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2eec2-0e8b-4681-97d3-0bc1958241bc admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:36.428813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-7cd2eec2-0e8b-4681-97d3-0bc1958241bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:36.444080 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2eec2-0e8b-4681-97d3-0bc1958241bc admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:36.449706 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:36.502062 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c296f-e0d0-475f-8101-9ee73c08f80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:36.502927 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c296f-e0d0-475f-8101-9ee73c08f80d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:36.503209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c296f-e0d0-475f-8101-9ee73c08f80d admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:36.503483 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c296f-e0d0-475f-8101-9ee73c08f80d admin admin] PATH_INFO: `/v3/projects/f6194539d3fb44549b899ca399f342a4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:36.505375 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c296f-e0d0-475f-8101-9ee73c08f80d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6194539d3fb44549b899ca399f342a4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:36.507390 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c296f-e0d0-475f-8101-9ee73c08f80d admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:36.604112 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e36c296f-e0d0-475f-8101-9ee73c08f80d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6194539d3fb44549b899ca399f342a4', 'request_id': 'req-e36c296f-e0d0-475f-8101-9ee73c08f80d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:36.610365 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e36c296f-e0d0-475f-8101-9ee73c08f80d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6194539d3fb44549b899ca399f342a4', 'request_id': 'req-e36c296f-e0d0-475f-8101-9ee73c08f80d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:36.616712 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e36c296f-e0d0-475f-8101-9ee73c08f80d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6194539d3fb44549b899ca399f342a4', 'request_id': 'req-e36c296f-e0d0-475f-8101-9ee73c08f80d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:36.715779 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e36c296f-e0d0-475f-8101-9ee73c08f80d admin admin] The token cache is being invalidate because project f6194539d3fb44549b899ca399f342a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:56:36.716004 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e36c296f-e0d0-475f-8101-9ee73c08f80d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:36.716769 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-e36c296f-e0d0-475f-8101-9ee73c08f80d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:36.718337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1410/2791] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:56:36 2022] DELETE /identity/v3/projects/f6194539d3fb44549b899ca399f342a4 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:36.725316 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582876d7-f976-41dc-b9a7-b152fdc9165c None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:36.725772 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582876d7-f976-41dc-b9a7-b152fdc9165c None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:36.725885 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582876d7-f976-41dc-b9a7-b152fdc9165c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:36.745271 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-582876d7-f976-41dc-b9a7-b152fdc9165c None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:36.746547 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-582876d7-f976-41dc-b9a7-b152fdc9165c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:36.806527 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1382/2792] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:36.814848 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9cf21-db2a-4808-bf78-a10e81cf84d4 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:36.815141 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9cf21-db2a-4808-bf78-a10e81cf84d4 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:36.815426 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9cf21-db2a-4808-bf78-a10e81cf84d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:36.834441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-eed9cf21-db2a-4808-bf78-a10e81cf84d4 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:36.835698 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-eed9cf21-db2a-4808-bf78-a10e81cf84d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:36.843701 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1411/2793] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:36.849155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582876d7-f976-41dc-b9a7-b152fdc9165c None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:36.858783 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582876d7-f976-41dc-b9a7-b152fdc9165c None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:36.863694 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716d339-0e19-495c-be24-1b3569a5748b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:36.865080 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716d339-0e19-495c-be24-1b3569a5748b admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:36.865367 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716d339-0e19-495c-be24-1b3569a5748b admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:36.865649 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716d339-0e19-495c-be24-1b3569a5748b admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:36.866379 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716d339-0e19-495c-be24-1b3569a5748b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:56:36.866687 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716d339-0e19-495c-be24-1b3569a5748b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:36.868292 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716d339-0e19-495c-be24-1b3569a5748b admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:36.880451 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1383/2794] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:56:36.885125 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9cf21-db2a-4808-bf78-a10e81cf84d4 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:36.894491 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9cf21-db2a-4808-bf78-a10e81cf84d4 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:36.899808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f5a23-eacc-4272-93af-ef8059236efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:36.900567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f5a23-eacc-4272-93af-ef8059236efb admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:36.900896 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f5a23-eacc-4272-93af-ef8059236efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:36.901156 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f5a23-eacc-4272-93af-ef8059236efb admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:36.901846 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f5a23-eacc-4272-93af-ef8059236efb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:36.903737 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f5a23-eacc-4272-93af-ef8059236efb admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:36.988107 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1412/2795] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:56:36 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:56:36.993385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716d339-0e19-495c-be24-1b3569a5748b admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:37.003477 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716d339-0e19-495c-be24-1b3569a5748b admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:37.007969 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:37.063241 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56801b7-8fbb-4c4b-843d-66f1981bf3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:37.063914 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56801b7-8fbb-4c4b-843d-66f1981bf3d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:37.064201 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56801b7-8fbb-4c4b-843d-66f1981bf3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:37.064482 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56801b7-8fbb-4c4b-843d-66f1981bf3d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:37.065197 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56801b7-8fbb-4c4b-843d-66f1981bf3d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:37.067746 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56801b7-8fbb-4c4b-843d-66f1981bf3d1 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:37.145061 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1384/2796] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:56:36 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:56:37.149208 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f5a23-eacc-4272-93af-ef8059236efb admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:37.159625 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f5a23-eacc-4272-93af-ef8059236efb admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:37.164459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc176687-7b3a-4d9b-97d4-f001b998dc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:37.165146 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc176687-7b3a-4d9b-97d4-f001b998dc3e admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:37.165429 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc176687-7b3a-4d9b-97d4-f001b998dc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:37.165703 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc176687-7b3a-4d9b-97d4-f001b998dc3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:37.166427 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc176687-7b3a-4d9b-97d4-f001b998dc3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:37.168010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc176687-7b3a-4d9b-97d4-f001b998dc3e admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:37.179111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1413/2797] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:56:37.183092 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56801b7-8fbb-4c4b-843d-66f1981bf3d1 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:37.193792 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56801b7-8fbb-4c4b-843d-66f1981bf3d1 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:37.198362 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad29ef9-36aa-491e-9d75-7830b26e172e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:37.199011 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad29ef9-36aa-491e-9d75-7830b26e172e admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:37.199295 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad29ef9-36aa-491e-9d75-7830b26e172e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:37.199572 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad29ef9-36aa-491e-9d75-7830b26e172e admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:37.200291 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad29ef9-36aa-491e-9d75-7830b26e172e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:37.201925 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad29ef9-36aa-491e-9d75-7830b26e172e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:37.211765 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1385/2798] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:56:37.215783 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc176687-7b3a-4d9b-97d4-f001b998dc3e admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:37.225853 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc176687-7b3a-4d9b-97d4-f001b998dc3e admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:37.230396 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8620e74-3d5d-48ea-b2f5-829bdbf16fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:37.231270 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8620e74-3d5d-48ea-b2f5-829bdbf16fac admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:37.231555 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8620e74-3d5d-48ea-b2f5-829bdbf16fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:37.231830 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8620e74-3d5d-48ea-b2f5-829bdbf16fac admin admin] PATH_INFO: `/v3/projects/504a89cc4d814c0184060efcaf7b2858/users/4d5e0c0bcd4a4f15ad8f76637d3e3869/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:37.246865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8620e74-3d5d-48ea-b2f5-829bdbf16fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=504a89cc4d814c0184060efcaf7b2858, user_id=4d5e0c0bcd4a4f15ad8f76637d3e3869, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:37.249475 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8620e74-3d5d-48ea-b2f5-829bdbf16fac admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:37.324802 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1414/2799] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:56:37 2022] PUT /identity/v3/projects/504a89cc4d814c0184060efcaf7b2858/users/4d5e0c0bcd4a4f15ad8f76637d3e3869/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:37.336736 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89aecc-2073-4944-acb7-b2b2366c20f3 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:37.339471 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89aecc-2073-4944-acb7-b2b2366c20f3 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:37.339471 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89aecc-2073-4944-acb7-b2b2366c20f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:37.363976 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-bf89aecc-2073-4944-acb7-b2b2366c20f3 None None] MFA Rules not processed for user `4d5e0c0bcd4a4f15ad8f76637d3e3869`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:37.368160 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-bf89aecc-2073-4944-acb7-b2b2366c20f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:37.427505 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1386/2800] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:37.438792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8620e74-3d5d-48ea-b2f5-829bdbf16fac admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:37.440459 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e8620e74-3d5d-48ea-b2f5-829bdbf16fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:37.480549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:37.497116 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8620e74-3d5d-48ea-b2f5-829bdbf16fac admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:37.500828 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b63be-cf2b-47a6-9e36-4cc9e8eba34a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:37.501469 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b63be-cf2b-47a6-9e36-4cc9e8eba34a service cinder] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:37.501755 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b63be-cf2b-47a6-9e36-4cc9e8eba34a service cinder] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:37.502033 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b63be-cf2b-47a6-9e36-4cc9e8eba34a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:37.511049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b63be-cf2b-47a6-9e36-4cc9e8eba34a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:37.512489 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b63be-cf2b-47a6-9e36-4cc9e8eba34a service cinder] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:37.522316 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1415/2801] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:56:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:56:37.541434 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89aecc-2073-4944-acb7-b2b2366c20f3 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:37.602468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89aecc-2073-4944-acb7-b2b2366c20f3 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:37.606913 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23814c-21da-44b1-8168-3f8714af5186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:37.607776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23814c-21da-44b1-8168-3f8714af5186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:37.608059 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23814c-21da-44b1-8168-3f8714af5186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:37.608338 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23814c-21da-44b1-8168-3f8714af5186 admin admin] PATH_INFO: `/v3/users/4d5e0c0bcd4a4f15ad8f76637d3e3869` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:37.609542 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23814c-21da-44b1-8168-3f8714af5186 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d5e0c0bcd4a4f15ad8f76637d3e3869)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:37.611485 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23814c-21da-44b1-8168-3f8714af5186 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:37.626780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:37.811816 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ad23814c-21da-44b1-8168-3f8714af5186 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d5e0c0bcd4a4f15ad8f76637d3e3869'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:37.824963 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ad23814c-21da-44b1-8168-3f8714af5186 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d5e0c0bcd4a4f15ad8f76637d3e3869'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:37.906846 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ad23814c-21da-44b1-8168-3f8714af5186 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d5e0c0bcd4a4f15ad8f76637d3e3869'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:37.907471 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ad23814c-21da-44b1-8168-3f8714af5186 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d5e0c0bcd4a4f15ad8f76637d3e3869'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:37.908122 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-ad23814c-21da-44b1-8168-3f8714af5186 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d5e0c0bcd4a4f15ad8f76637d3e3869'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:37.921036 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1387/2802] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:56:37 2022] DELETE /identity/v3/users/4d5e0c0bcd4a4f15ad8f76637d3e3869 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:37.925487 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bece9e-27c5-4d61-a3df-a87ab04c4724 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:37.925789 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bece9e-27c5-4d61-a3df-a87ab04c4724 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:37.926112 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bece9e-27c5-4d61-a3df-a87ab04c4724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:37.944106 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-25bece9e-27c5-4d61-a3df-a87ab04c4724 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:37.945332 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-25bece9e-27c5-4d61-a3df-a87ab04c4724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:38.003853 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1416/2803] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:38.013938 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23814c-21da-44b1-8168-3f8714af5186 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:38.015323 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ad23814c-21da-44b1-8168-3f8714af5186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:38.075880 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23814c-21da-44b1-8168-3f8714af5186 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:38.079533 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be16ba-a5c0-4729-9564-d32e45d1e771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:38.081158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be16ba-a5c0-4729-9564-d32e45d1e771 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:38.081158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be16ba-a5c0-4729-9564-d32e45d1e771 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:38.081158 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be16ba-a5c0-4729-9564-d32e45d1e771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:38.090413 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be16ba-a5c0-4729-9564-d32e45d1e771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:38.091904 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be16ba-a5c0-4729-9564-d32e45d1e771 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:38.102482 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1388/2804] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:56:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:56:38.514433 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bece9e-27c5-4d61-a3df-a87ab04c4724 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:38.517557 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-25bece9e-27c5-4d61-a3df-a87ab04c4724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:38.530227 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bece9e-27c5-4d61-a3df-a87ab04c4724 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:38.536777 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb438b54-9922-44ea-98a9-f85ff8009205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:38.537731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb438b54-9922-44ea-98a9-f85ff8009205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:38.537933 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb438b54-9922-44ea-98a9-f85ff8009205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:38.538251 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb438b54-9922-44ea-98a9-f85ff8009205 admin admin] PATH_INFO: `/v3/projects/504a89cc4d814c0184060efcaf7b2858` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:38.539508 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb438b54-9922-44ea-98a9-f85ff8009205 admin admin] RBAC: Authorizing `identity:delete_project(project_id=504a89cc4d814c0184060efcaf7b2858)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:38.541480 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb438b54-9922-44ea-98a9-f85ff8009205 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:38.645162 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-eb438b54-9922-44ea-98a9-f85ff8009205 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '504a89cc4d814c0184060efcaf7b2858', 'request_id': 'req-eb438b54-9922-44ea-98a9-f85ff8009205'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:38.651359 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-eb438b54-9922-44ea-98a9-f85ff8009205 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '504a89cc4d814c0184060efcaf7b2858', 'request_id': 'req-eb438b54-9922-44ea-98a9-f85ff8009205'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:38.656987 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-eb438b54-9922-44ea-98a9-f85ff8009205 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '504a89cc4d814c0184060efcaf7b2858', 'request_id': 'req-eb438b54-9922-44ea-98a9-f85ff8009205'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:38.746084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-eb438b54-9922-44ea-98a9-f85ff8009205 admin admin] The token cache is being invalidate because project 504a89cc4d814c0184060efcaf7b2858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:56:38.746375 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-eb438b54-9922-44ea-98a9-f85ff8009205 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:38.747005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-eb438b54-9922-44ea-98a9-f85ff8009205 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:38.748543 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1417/2805] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:56:38 2022] DELETE /identity/v3/projects/504a89cc4d814c0184060efcaf7b2858 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:38.753673 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba2f4d-475b-4d18-8067-f349ff2a6c72 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:38.753968 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba2f4d-475b-4d18-8067-f349ff2a6c72 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:38.754249 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba2f4d-475b-4d18-8067-f349ff2a6c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:38.775003 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-b5ba2f4d-475b-4d18-8067-f349ff2a6c72 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:38.776204 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-b5ba2f4d-475b-4d18-8067-f349ff2a6c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:38.836071 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1389/2806] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:38.845360 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68481d-96b5-427c-b8ef-f9c798e0158d None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:38.845650 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68481d-96b5-427c-b8ef-f9c798e0158d None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:38.845929 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68481d-96b5-427c-b8ef-f9c798e0158d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:38.865388 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-ac68481d-96b5-427c-b8ef-f9c798e0158d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:38.866590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ac68481d-96b5-427c-b8ef-f9c798e0158d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:38.874351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1418/2807] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:38.878618 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba2f4d-475b-4d18-8067-f349ff2a6c72 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:38.888887 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba2f4d-475b-4d18-8067-f349ff2a6c72 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:38.893371 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ad2c9-9728-4b70-9cbe-96f0d181924a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:38.894030 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ad2c9-9728-4b70-9cbe-96f0d181924a admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:38.894318 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ad2c9-9728-4b70-9cbe-96f0d181924a admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:38.894598 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ad2c9-9728-4b70-9cbe-96f0d181924a admin admin] PATH_INFO: `/v3/domains` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:38.896101 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ad2c9-9728-4b70-9cbe-96f0d181924a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80519) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:56:38.896342 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ad2c9-9728-4b70-9cbe-96f0d181924a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:38.897977 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ad2c9-9728-4b70-9cbe-96f0d181924a admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:38.909363 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1390/2808] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:56:38.913651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68481d-96b5-427c-b8ef-f9c798e0158d None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:38.922709 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68481d-96b5-427c-b8ef-f9c798e0158d None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:38.927369 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b4b17-4a05-442b-97da-983218bfb88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:38.928007 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b4b17-4a05-442b-97da-983218bfb88c admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:38.928289 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b4b17-4a05-442b-97da-983218bfb88c admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:38.928564 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b4b17-4a05-442b-97da-983218bfb88c admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:38.929279 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b4b17-4a05-442b-97da-983218bfb88c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:38.931144 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b4b17-4a05-442b-97da-983218bfb88c admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:39.009351 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1419/2809] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:56:38 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:56:39.013539 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ad2c9-9728-4b70-9cbe-96f0d181924a admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:39.024270 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ad2c9-9728-4b70-9cbe-96f0d181924a admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:39.044253 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:39.081667 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ec459-23d0-4105-ad5e-6461c5d170a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:39.082338 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ec459-23d0-4105-ad5e-6461c5d170a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:39.083188 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ec459-23d0-4105-ad5e-6461c5d170a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:39.083469 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ec459-23d0-4105-ad5e-6461c5d170a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:39.084155 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ec459-23d0-4105-ad5e-6461c5d170a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:39.086231 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ec459-23d0-4105-ad5e-6461c5d170a6 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:39.169468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1391/2810] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:56:39 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:56:39.173702 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b4b17-4a05-442b-97da-983218bfb88c admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:39.184597 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b4b17-4a05-442b-97da-983218bfb88c admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:39.189152 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6370874-b2c6-419a-8b57-2da3322737d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:39.189808 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6370874-b2c6-419a-8b57-2da3322737d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:39.190094 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6370874-b2c6-419a-8b57-2da3322737d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:39.190374 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6370874-b2c6-419a-8b57-2da3322737d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:39.191133 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6370874-b2c6-419a-8b57-2da3322737d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:39.193596 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6370874-b2c6-419a-8b57-2da3322737d2 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:39.204004 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1420/2811] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:56:39.207862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ec459-23d0-4105-ad5e-6461c5d170a6 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:39.217823 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ec459-23d0-4105-ad5e-6461c5d170a6 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:39.222696 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82417e-5ca1-4d73-b003-24ac4533d5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:39.223493 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82417e-5ca1-4d73-b003-24ac4533d5ae admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:39.223782 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82417e-5ca1-4d73-b003-24ac4533d5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:39.224061 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82417e-5ca1-4d73-b003-24ac4533d5ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:39.224828 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82417e-5ca1-4d73-b003-24ac4533d5ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:39.226412 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82417e-5ca1-4d73-b003-24ac4533d5ae admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:39.236241 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1392/2812] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:56:39.241002 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6370874-b2c6-419a-8b57-2da3322737d2 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:39.251224 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6370874-b2c6-419a-8b57-2da3322737d2 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:39.256843 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd240208-966f-4498-88b5-2170d6f8ea39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:39.257741 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd240208-966f-4498-88b5-2170d6f8ea39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:39.258026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd240208-966f-4498-88b5-2170d6f8ea39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:39.258299 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd240208-966f-4498-88b5-2170d6f8ea39 admin admin] PATH_INFO: `/v3/projects/e6c069e0556244d6884fa47fe3d60376/users/bc7cbf3a920a454fae335c932ee0ffd2/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:39.273130 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd240208-966f-4498-88b5-2170d6f8ea39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6c069e0556244d6884fa47fe3d60376, user_id=bc7cbf3a920a454fae335c932ee0ffd2, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:39.275769 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd240208-966f-4498-88b5-2170d6f8ea39 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:39.341739 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:39.347535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1421/2813] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:56:39 2022] PUT /identity/v3/projects/e6c069e0556244d6884fa47fe3d60376/users/bc7cbf3a920a454fae335c932ee0ffd2/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:39.357647 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58fb07-2beb-46c4-91b4-7c2c454c7d01 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:39.357944 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58fb07-2beb-46c4-91b4-7c2c454c7d01 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:39.358271 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58fb07-2beb-46c4-91b4-7c2c454c7d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:39.387581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-9d58fb07-2beb-46c4-91b4-7c2c454c7d01 None None] MFA Rules not processed for user `bc7cbf3a920a454fae335c932ee0ffd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:39.389596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-9d58fb07-2beb-46c4-91b4-7c2c454c7d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:39.453846 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1393/2814] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:39.458435 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd240208-966f-4498-88b5-2170d6f8ea39 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:39.520271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd240208-966f-4498-88b5-2170d6f8ea39 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:39.525184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddeb5b-28b3-4f27-af46-61840ec196eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:39.525844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddeb5b-28b3-4f27-af46-61840ec196eb admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:39.526134 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddeb5b-28b3-4f27-af46-61840ec196eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:39.526416 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddeb5b-28b3-4f27-af46-61840ec196eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:39.527133 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddeb5b-28b3-4f27-af46-61840ec196eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:39.529113 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddeb5b-28b3-4f27-af46-61840ec196eb admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:39.608893 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1422/2815] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:56:39 2022] POST /identity/v3/projects => generated 375 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:56:39.612915 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58fb07-2beb-46c4-91b4-7c2c454c7d01 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:39.624406 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58fb07-2beb-46c4-91b4-7c2c454c7d01 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:39.681001 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe8655-e67d-4f49-99b1-613bed6997eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:39.681673 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe8655-e67d-4f49-99b1-613bed6997eb admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:39.681993 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe8655-e67d-4f49-99b1-613bed6997eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:39.682277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe8655-e67d-4f49-99b1-613bed6997eb admin admin] PATH_INFO: `/v3/users` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:39.682975 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe8655-e67d-4f49-99b1-613bed6997eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:39.684937 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe8655-e67d-4f49-99b1-613bed6997eb admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:39.809712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1394/2816] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:56:39 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:56:39.813867 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddeb5b-28b3-4f27-af46-61840ec196eb admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:39.824856 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddeb5b-28b3-4f27-af46-61840ec196eb admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:39.829610 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d4419-0808-40f3-94a8-5c7176b90fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:39.830303 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d4419-0808-40f3-94a8-5c7176b90fdd admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:39.830592 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d4419-0808-40f3-94a8-5c7176b90fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:39.830872 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d4419-0808-40f3-94a8-5c7176b90fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:39.831603 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d4419-0808-40f3-94a8-5c7176b90fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:39.834028 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d4419-0808-40f3-94a8-5c7176b90fdd admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:39.845494 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1423/2817] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:56:39.849144 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe8655-e67d-4f49-99b1-613bed6997eb admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:39.859053 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe8655-e67d-4f49-99b1-613bed6997eb admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:39.863694 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f2af7-a541-4415-a464-7dcde97ee292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:39.865304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f2af7-a541-4415-a464-7dcde97ee292 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:39.865304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f2af7-a541-4415-a464-7dcde97ee292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:39.865304 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f2af7-a541-4415-a464-7dcde97ee292 admin admin] PATH_INFO: `/v3/projects/e1bea5e96fef422bae9c973ec8f69af8/users/e27b8884637a49a4bc914f52dbcea0bc/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:39.880767 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f2af7-a541-4415-a464-7dcde97ee292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1bea5e96fef422bae9c973ec8f69af8, user_id=e27b8884637a49a4bc914f52dbcea0bc, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:39.883482 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f2af7-a541-4415-a464-7dcde97ee292 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:39.947664 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:39.953114 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1395/2818] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:56:39 2022] PUT /identity/v3/projects/e1bea5e96fef422bae9c973ec8f69af8/users/e27b8884637a49a4bc914f52dbcea0bc/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:39.957188 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d4419-0808-40f3-94a8-5c7176b90fdd admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:39.967499 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d4419-0808-40f3-94a8-5c7176b90fdd admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:40.025406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acab556-43c9-4c1b-aa08-04adb1fc3e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:40.026454 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acab556-43c9-4c1b-aa08-04adb1fc3e16 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:40.027157 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acab556-43c9-4c1b-aa08-04adb1fc3e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:40.027438 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acab556-43c9-4c1b-aa08-04adb1fc3e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:40.028164 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acab556-43c9-4c1b-aa08-04adb1fc3e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:40.030578 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acab556-43c9-4c1b-aa08-04adb1fc3e16 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:40.041589 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1424/2819] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:56:40.045696 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f2af7-a541-4415-a464-7dcde97ee292 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:40.057138 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f2af7-a541-4415-a464-7dcde97ee292 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:40.061947 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91a5b4-cc17-48ad-b793-2465ce09e4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:40.062773 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91a5b4-cc17-48ad-b793-2465ce09e4d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:40.063062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91a5b4-cc17-48ad-b793-2465ce09e4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:40.063342 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91a5b4-cc17-48ad-b793-2465ce09e4d2 admin admin] PATH_INFO: `/v3/projects/e1bea5e96fef422bae9c973ec8f69af8/users/e27b8884637a49a4bc914f52dbcea0bc/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:40.065463 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91a5b4-cc17-48ad-b793-2465ce09e4d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1bea5e96fef422bae9c973ec8f69af8, user_id=e27b8884637a49a4bc914f52dbcea0bc, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:40.069191 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91a5b4-cc17-48ad-b793-2465ce09e4d2 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:40.079313 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1396/2820] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:56:40 2022] PUT /identity/v3/projects/e1bea5e96fef422bae9c973ec8f69af8/users/e27b8884637a49a4bc914f52dbcea0bc/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:40.089529 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ff4e0-caa1-40d3-818f-fc8e7fe7ab17 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:40.089824 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ff4e0-caa1-40d3-818f-fc8e7fe7ab17 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:40.090167 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ff4e0-caa1-40d3-818f-fc8e7fe7ab17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:40.117127 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-785ff4e0-caa1-40d3-818f-fc8e7fe7ab17 None None] MFA Rules not processed for user `e27b8884637a49a4bc914f52dbcea0bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:40.118339 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-785ff4e0-caa1-40d3-818f-fc8e7fe7ab17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:40.175688 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:40.186222 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1425/2821] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:40.198029 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91a5b4-cc17-48ad-b793-2465ce09e4d2 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:40.199532 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-de91a5b4-cc17-48ad-b793-2465ce09e4d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:40.258087 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91a5b4-cc17-48ad-b793-2465ce09e4d2 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:40.261854 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4ee0b-728f-4672-b185-d0e150d4430b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:40.262537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4ee0b-728f-4672-b185-d0e150d4430b service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:40.262851 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4ee0b-728f-4672-b185-d0e150d4430b service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:40.263137 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4ee0b-728f-4672-b185-d0e150d4430b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:40.322024 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4ee0b-728f-4672-b185-d0e150d4430b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:40.323551 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4ee0b-728f-4672-b185-d0e150d4430b service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:40.333250 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1397/2822] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:56:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:56:43.356080 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ff4e0-caa1-40d3-818f-fc8e7fe7ab17 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:43.417839 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ff4e0-caa1-40d3-818f-fc8e7fe7ab17 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:43.422380 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1679803-903c-4e9f-a074-57ef4207d9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:43.423248 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1679803-903c-4e9f-a074-57ef4207d9ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:43.423538 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1679803-903c-4e9f-a074-57ef4207d9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:43.423818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1679803-903c-4e9f-a074-57ef4207d9ed admin admin] PATH_INFO: `/v3/users/bc7cbf3a920a454fae335c932ee0ffd2` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:43.425036 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1679803-903c-4e9f-a074-57ef4207d9ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc7cbf3a920a454fae335c932ee0ffd2)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:43.427047 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1679803-903c-4e9f-a074-57ef4207d9ed admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:43.620567 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a1679803-903c-4e9f-a074-57ef4207d9ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc7cbf3a920a454fae335c932ee0ffd2'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:43.632830 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a1679803-903c-4e9f-a074-57ef4207d9ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc7cbf3a920a454fae335c932ee0ffd2'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:43.633508 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a1679803-903c-4e9f-a074-57ef4207d9ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc7cbf3a920a454fae335c932ee0ffd2'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:43.634111 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a1679803-903c-4e9f-a074-57ef4207d9ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc7cbf3a920a454fae335c932ee0ffd2'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:43.641071 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a1679803-903c-4e9f-a074-57ef4207d9ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc7cbf3a920a454fae335c932ee0ffd2'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:43.715955 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1426/2823] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:56:43 2022] DELETE /identity/v3/users/bc7cbf3a920a454fae335c932ee0ffd2 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:43.720482 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77588f3c-a7f0-4194-a4ff-485c16724db3 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:43.720996 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77588f3c-a7f0-4194-a4ff-485c16724db3 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:43.721377 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77588f3c-a7f0-4194-a4ff-485c16724db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:43.739973 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-77588f3c-a7f0-4194-a4ff-485c16724db3 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:43.741196 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-77588f3c-a7f0-4194-a4ff-485c16724db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:43.800201 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1398/2824] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:43.810256 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1679803-903c-4e9f-a074-57ef4207d9ed admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:43.811684 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-a1679803-903c-4e9f-a074-57ef4207d9ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:43.869215 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1679803-903c-4e9f-a074-57ef4207d9ed admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:43.872924 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937a908-84a7-423b-b15b-479606f556a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:43.873507 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937a908-84a7-423b-b15b-479606f556a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:43.873793 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937a908-84a7-423b-b15b-479606f556a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:43.874076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937a908-84a7-423b-b15b-479606f556a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:43.883880 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937a908-84a7-423b-b15b-479606f556a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:43.885337 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937a908-84a7-423b-b15b-479606f556a9 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:43.894772 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1427/2825] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:56:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:56:44.323469 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77588f3c-a7f0-4194-a4ff-485c16724db3 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:44.324909 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-77588f3c-a7f0-4194-a4ff-485c16724db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:44.337019 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77588f3c-a7f0-4194-a4ff-485c16724db3 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:44.341675 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc9842-7ea0-4ad8-b8a0-cc56d5f57071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:44.342536 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc9842-7ea0-4ad8-b8a0-cc56d5f57071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:44.342818 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc9842-7ea0-4ad8-b8a0-cc56d5f57071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:44.343092 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc9842-7ea0-4ad8-b8a0-cc56d5f57071 admin admin] PATH_INFO: `/v3/projects/e6c069e0556244d6884fa47fe3d60376` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:44.344273 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc9842-7ea0-4ad8-b8a0-cc56d5f57071 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6c069e0556244d6884fa47fe3d60376)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:44.346747 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc9842-7ea0-4ad8-b8a0-cc56d5f57071 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:44.440206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:44.449413 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d8bc9842-7ea0-4ad8-b8a0-cc56d5f57071 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6c069e0556244d6884fa47fe3d60376', 'request_id': 'req-d8bc9842-7ea0-4ad8-b8a0-cc56d5f57071'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:44.529135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d8bc9842-7ea0-4ad8-b8a0-cc56d5f57071 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6c069e0556244d6884fa47fe3d60376', 'request_id': 'req-d8bc9842-7ea0-4ad8-b8a0-cc56d5f57071'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:44.536341 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d8bc9842-7ea0-4ad8-b8a0-cc56d5f57071 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6c069e0556244d6884fa47fe3d60376', 'request_id': 'req-d8bc9842-7ea0-4ad8-b8a0-cc56d5f57071'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:44.547064 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d8bc9842-7ea0-4ad8-b8a0-cc56d5f57071 admin admin] The token cache is being invalidate because project e6c069e0556244d6884fa47fe3d60376 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:56:44.547353 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d8bc9842-7ea0-4ad8-b8a0-cc56d5f57071 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:44.548016 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-d8bc9842-7ea0-4ad8-b8a0-cc56d5f57071 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:44.549593 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1399/2826] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:56:44 2022] DELETE /identity/v3/projects/e6c069e0556244d6884fa47fe3d60376 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:44.553795 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937a908-84a7-423b-b15b-479606f556a9 service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:44.563415 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937a908-84a7-423b-b15b-479606f556a9 service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:44.575039 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:44.619947 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b89b15-5a21-485b-b5e8-5c3de689f06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:44.620853 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b89b15-5a21-485b-b5e8-5c3de689f06d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:44.621142 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b89b15-5a21-485b-b5e8-5c3de689f06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:44.621418 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b89b15-5a21-485b-b5e8-5c3de689f06d admin admin] PATH_INFO: `/v3/users/e27b8884637a49a4bc914f52dbcea0bc` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:44.622604 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b89b15-5a21-485b-b5e8-5c3de689f06d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e27b8884637a49a4bc914f52dbcea0bc)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:44.624605 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b89b15-5a21-485b-b5e8-5c3de689f06d admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:44.832844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a9b89b15-5a21-485b-b5e8-5c3de689f06d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e27b8884637a49a4bc914f52dbcea0bc'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:44.845079 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a9b89b15-5a21-485b-b5e8-5c3de689f06d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e27b8884637a49a4bc914f52dbcea0bc'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:44.845756 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a9b89b15-5a21-485b-b5e8-5c3de689f06d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e27b8884637a49a4bc914f52dbcea0bc'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:44.846655 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a9b89b15-5a21-485b-b5e8-5c3de689f06d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e27b8884637a49a4bc914f52dbcea0bc'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:44.853578 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-a9b89b15-5a21-485b-b5e8-5c3de689f06d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e27b8884637a49a4bc914f52dbcea0bc'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:44.945069 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1428/2827] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:56:44 2022] DELETE /identity/v3/users/e27b8884637a49a4bc914f52dbcea0bc => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:45.330855 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc9842-7ea0-4ad8-b8a0-cc56d5f57071 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:45.332930 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-d8bc9842-7ea0-4ad8-b8a0-cc56d5f57071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:45.348775 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc9842-7ea0-4ad8-b8a0-cc56d5f57071 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:45.406340 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aeae61-a082-4e37-bee5-c81b2e15d8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:45.407198 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aeae61-a082-4e37-bee5-c81b2e15d8a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:45.407487 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aeae61-a082-4e37-bee5-c81b2e15d8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:45.407761 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aeae61-a082-4e37-bee5-c81b2e15d8a8 admin admin] PATH_INFO: `/v3/projects/e1bea5e96fef422bae9c973ec8f69af8` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:45.408952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aeae61-a082-4e37-bee5-c81b2e15d8a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1bea5e96fef422bae9c973ec8f69af8)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:45.411546 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aeae61-a082-4e37-bee5-c81b2e15d8a8 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:45.549992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-39aeae61-a082-4e37-bee5-c81b2e15d8a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1bea5e96fef422bae9c973ec8f69af8', 'request_id': 'req-39aeae61-a082-4e37-bee5-c81b2e15d8a8'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:45.624537 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-39aeae61-a082-4e37-bee5-c81b2e15d8a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1bea5e96fef422bae9c973ec8f69af8', 'request_id': 'req-39aeae61-a082-4e37-bee5-c81b2e15d8a8'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:45.631401 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-39aeae61-a082-4e37-bee5-c81b2e15d8a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1bea5e96fef422bae9c973ec8f69af8', 'request_id': 'req-39aeae61-a082-4e37-bee5-c81b2e15d8a8'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:45.641966 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-39aeae61-a082-4e37-bee5-c81b2e15d8a8 admin admin] The token cache is being invalidate because project e1bea5e96fef422bae9c973ec8f69af8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:56:45.642253 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-39aeae61-a082-4e37-bee5-c81b2e15d8a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:45.642888 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-39aeae61-a082-4e37-bee5-c81b2e15d8a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:45.644417 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1400/2828] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:56:45 2022] DELETE /identity/v3/projects/e1bea5e96fef422bae9c973ec8f69af8 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:45.650009 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53269dd7-987a-4637-a2ab-d4f82eb56756 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:45.650524 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53269dd7-987a-4637-a2ab-d4f82eb56756 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:45.650658 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53269dd7-987a-4637-a2ab-d4f82eb56756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:45.667884 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-53269dd7-987a-4637-a2ab-d4f82eb56756 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:45.669138 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-53269dd7-987a-4637-a2ab-d4f82eb56756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:45.727650 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1429/2829] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:45.736570 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab2b1e-1fa1-44f2-b483-39db1664ff46 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:45.737384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab2b1e-1fa1-44f2-b483-39db1664ff46 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:45.737384 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab2b1e-1fa1-44f2-b483-39db1664ff46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:45.755723 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-e8ab2b1e-1fa1-44f2-b483-39db1664ff46 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:45.756952 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e8ab2b1e-1fa1-44f2-b483-39db1664ff46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:45.764727 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1401/2830] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:45.769121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53269dd7-987a-4637-a2ab-d4f82eb56756 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:45.779982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53269dd7-987a-4637-a2ab-d4f82eb56756 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:45.784772 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491533b-33d8-48ed-b020-fad2ac5f7773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:45.785448 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491533b-33d8-48ed-b020-fad2ac5f7773 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:45.785735 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491533b-33d8-48ed-b020-fad2ac5f7773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:45.786014 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491533b-33d8-48ed-b020-fad2ac5f7773 admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:45.786773 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491533b-33d8-48ed-b020-fad2ac5f7773 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:56:45.787081 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491533b-33d8-48ed-b020-fad2ac5f7773 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:45.788715 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491533b-33d8-48ed-b020-fad2ac5f7773 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:45.800266 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1430/2831] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:56:45.804525 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab2b1e-1fa1-44f2-b483-39db1664ff46 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:45.813811 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab2b1e-1fa1-44f2-b483-39db1664ff46 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:45.819074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7ada7-e6a1-4f5a-b27a-ae239cdb1055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:45.819716 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7ada7-e6a1-4f5a-b27a-ae239cdb1055 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:45.820004 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7ada7-e6a1-4f5a-b27a-ae239cdb1055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:45.820283 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7ada7-e6a1-4f5a-b27a-ae239cdb1055 admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:45.821001 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7ada7-e6a1-4f5a-b27a-ae239cdb1055 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:45.822905 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7ada7-e6a1-4f5a-b27a-ae239cdb1055 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:45.929395 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1402/2832] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:56:45 2022] POST /identity/v3/projects => generated 371 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:56:45.933520 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491533b-33d8-48ed-b020-fad2ac5f7773 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:45.944132 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491533b-33d8-48ed-b020-fad2ac5f7773 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:45.962547 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:46.000368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246a390-7989-414e-ab73-68a2ff448a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:46.001061 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246a390-7989-414e-ab73-68a2ff448a4b admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:46.001350 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246a390-7989-414e-ab73-68a2ff448a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:46.001635 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246a390-7989-414e-ab73-68a2ff448a4b admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:46.002385 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246a390-7989-414e-ab73-68a2ff448a4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:46.005030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246a390-7989-414e-ab73-68a2ff448a4b admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:46.116493 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1431/2833] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:56:45 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:56:46.120660 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7ada7-e6a1-4f5a-b27a-ae239cdb1055 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:46.131659 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7ada7-e6a1-4f5a-b27a-ae239cdb1055 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:46.136857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7968aa1-3cc7-4a0a-a447-3e745f5414d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:46.137523 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7968aa1-3cc7-4a0a-a447-3e745f5414d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:46.137811 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7968aa1-3cc7-4a0a-a447-3e745f5414d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:46.138090 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7968aa1-3cc7-4a0a-a447-3e745f5414d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:46.138869 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7968aa1-3cc7-4a0a-a447-3e745f5414d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:46.140534 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7968aa1-3cc7-4a0a-a447-3e745f5414d4 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:46.151265 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1403/2834] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:56:46.155247 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246a390-7989-414e-ab73-68a2ff448a4b admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:46.165419 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246a390-7989-414e-ab73-68a2ff448a4b admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:46.170084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e797c-4166-427a-bfac-bd460349852a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:46.170740 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e797c-4166-427a-bfac-bd460349852a admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:46.171028 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e797c-4166-427a-bfac-bd460349852a admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:46.171308 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e797c-4166-427a-bfac-bd460349852a admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:46.172032 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e797c-4166-427a-bfac-bd460349852a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:46.173655 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e797c-4166-427a-bfac-bd460349852a admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:46.184262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1432/2835] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:56:46.188295 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7968aa1-3cc7-4a0a-a447-3e745f5414d4 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:46.199129 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7968aa1-3cc7-4a0a-a447-3e745f5414d4 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:46.203726 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a932c5-e141-46db-baad-d04d734b34d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:46.204617 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a932c5-e141-46db-baad-d04d734b34d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:46.204901 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a932c5-e141-46db-baad-d04d734b34d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:46.205186 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a932c5-e141-46db-baad-d04d734b34d7 admin admin] PATH_INFO: `/v3/projects/e3b7db7fb91c49e585cc128e266bd805/users/0ecf0e36974243aab352aa82f0131d07/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:46.216029 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:46.219861 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a932c5-e141-46db-baad-d04d734b34d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3b7db7fb91c49e585cc128e266bd805, user_id=0ecf0e36974243aab352aa82f0131d07, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:46.222834 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a932c5-e141-46db-baad-d04d734b34d7 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:46.278385 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1404/2836] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:56:46 2022] PUT /identity/v3/projects/e3b7db7fb91c49e585cc128e266bd805/users/0ecf0e36974243aab352aa82f0131d07/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:46.339630 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99535d6e-51d8-43c0-9b62-4417e64a1ffb None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:46.339930 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99535d6e-51d8-43c0-9b62-4417e64a1ffb None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:46.340216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99535d6e-51d8-43c0-9b62-4417e64a1ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:46.368825 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-99535d6e-51d8-43c0-9b62-4417e64a1ffb None None] MFA Rules not processed for user `0ecf0e36974243aab352aa82f0131d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:46.370084 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-99535d6e-51d8-43c0-9b62-4417e64a1ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:46.435693 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1433/2837] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:46.447255 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a932c5-e141-46db-baad-d04d734b34d7 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:46.449018 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-77a932c5-e141-46db-baad-d04d734b34d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:46.507438 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a932c5-e141-46db-baad-d04d734b34d7 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:46.511148 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f794ea1-d110-4a29-b179-c23894c3d67b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:46.511799 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f794ea1-d110-4a29-b179-c23894c3d67b service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:46.512088 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f794ea1-d110-4a29-b179-c23894c3d67b service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:46.512369 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f794ea1-d110-4a29-b179-c23894c3d67b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:46.520933 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f794ea1-d110-4a29-b179-c23894c3d67b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:46.522376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f794ea1-d110-4a29-b179-c23894c3d67b service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:46.531280 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1405/2838] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:56:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:56:46.556363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99535d6e-51d8-43c0-9b62-4417e64a1ffb None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:46.618775 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99535d6e-51d8-43c0-9b62-4417e64a1ffb None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:46.623827 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952c7dc-6ca2-4a2b-87c3-79f019775ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:46.624728 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952c7dc-6ca2-4a2b-87c3-79f019775ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:46.625010 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952c7dc-6ca2-4a2b-87c3-79f019775ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:46.625282 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952c7dc-6ca2-4a2b-87c3-79f019775ee9 admin admin] PATH_INFO: `/v3/users/0ecf0e36974243aab352aa82f0131d07` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:46.626497 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952c7dc-6ca2-4a2b-87c3-79f019775ee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ecf0e36974243aab352aa82f0131d07)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:46.628597 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952c7dc-6ca2-4a2b-87c3-79f019775ee9 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:46.817007 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:46.828964 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c952c7dc-6ca2-4a2b-87c3-79f019775ee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ecf0e36974243aab352aa82f0131d07'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:46.840957 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c952c7dc-6ca2-4a2b-87c3-79f019775ee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ecf0e36974243aab352aa82f0131d07'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:46.841770 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c952c7dc-6ca2-4a2b-87c3-79f019775ee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ecf0e36974243aab352aa82f0131d07'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:46.842398 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c952c7dc-6ca2-4a2b-87c3-79f019775ee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ecf0e36974243aab352aa82f0131d07'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:46.850536 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-c952c7dc-6ca2-4a2b-87c3-79f019775ee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ecf0e36974243aab352aa82f0131d07'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:46.940208 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1434/2839] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:56:46 2022] DELETE /identity/v3/users/0ecf0e36974243aab352aa82f0131d07 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:46.945192 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb4e41-4a2f-4d66-b841-b5dd8a9a079d None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:46.945495 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb4e41-4a2f-4d66-b841-b5dd8a9a079d None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:46.945780 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb4e41-4a2f-4d66-b841-b5dd8a9a079d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:46.963621 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-4bcb4e41-4a2f-4d66-b841-b5dd8a9a079d None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:46.964985 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-4bcb4e41-4a2f-4d66-b841-b5dd8a9a079d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:47.023478 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1406/2840] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:47.032949 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952c7dc-6ca2-4a2b-87c3-79f019775ee9 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:47.034355 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-c952c7dc-6ca2-4a2b-87c3-79f019775ee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:47.090863 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952c7dc-6ca2-4a2b-87c3-79f019775ee9 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:47.094553 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18525312-af91-4f15-b9ce-d24e2db2e079 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:47.095196 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18525312-af91-4f15-b9ce-d24e2db2e079 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:47.095490 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18525312-af91-4f15-b9ce-d24e2db2e079 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:47.095767 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18525312-af91-4f15-b9ce-d24e2db2e079 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:47.105082 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18525312-af91-4f15-b9ce-d24e2db2e079 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:47.106511 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18525312-af91-4f15-b9ce-d24e2db2e079 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:47.116078 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1435/2841] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:56:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:56:47.542323 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb4e41-4a2f-4d66-b841-b5dd8a9a079d None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:47.543882 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-4bcb4e41-4a2f-4d66-b841-b5dd8a9a079d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:47.560074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb4e41-4a2f-4d66-b841-b5dd8a9a079d None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:47.561675 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734cddf-1fc6-418c-9c06-6067e6a0ed7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:47.562548 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734cddf-1fc6-418c-9c06-6067e6a0ed7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:47.562894 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734cddf-1fc6-418c-9c06-6067e6a0ed7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:47.563176 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734cddf-1fc6-418c-9c06-6067e6a0ed7b admin admin] PATH_INFO: `/v3/projects/e3b7db7fb91c49e585cc128e266bd805` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:47.564794 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f734cddf-1fc6-418c-9c06-6067e6a0ed7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3b7db7fb91c49e585cc128e266bd805)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:47.567488 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f734cddf-1fc6-418c-9c06-6067e6a0ed7b admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:47.693968 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f734cddf-1fc6-418c-9c06-6067e6a0ed7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3b7db7fb91c49e585cc128e266bd805', 'request_id': 'req-f734cddf-1fc6-418c-9c06-6067e6a0ed7b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:47.769095 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f734cddf-1fc6-418c-9c06-6067e6a0ed7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3b7db7fb91c49e585cc128e266bd805', 'request_id': 'req-f734cddf-1fc6-418c-9c06-6067e6a0ed7b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:47.775998 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f734cddf-1fc6-418c-9c06-6067e6a0ed7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3b7db7fb91c49e585cc128e266bd805', 'request_id': 'req-f734cddf-1fc6-418c-9c06-6067e6a0ed7b'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:47.787057 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f734cddf-1fc6-418c-9c06-6067e6a0ed7b admin admin] The token cache is being invalidate because project e3b7db7fb91c49e585cc128e266bd805 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:56:47.787351 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f734cddf-1fc6-418c-9c06-6067e6a0ed7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:47.787993 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-f734cddf-1fc6-418c-9c06-6067e6a0ed7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:56:47.789579 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1407/2842] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:56:47 2022] DELETE /identity/v3/projects/e3b7db7fb91c49e585cc128e266bd805 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:47.794462 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a313b1-f137-4896-a38a-11cd3963ad5b None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:47.794782 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a313b1-f137-4896-a38a-11cd3963ad5b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:47.796006 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a313b1-f137-4896-a38a-11cd3963ad5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:47.813157 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-11a313b1-f137-4896-a38a-11cd3963ad5b None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:47.814442 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-11a313b1-f137-4896-a38a-11cd3963ad5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:47.878063 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1436/2843] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:47.885874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c8c58-3daf-4f69-a4f2-b7f43276b06b None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:47.886171 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c8c58-3daf-4f69-a4f2-b7f43276b06b None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:47.886455 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c8c58-3daf-4f69-a4f2-b7f43276b06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:47.904650 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-941c8c58-3daf-4f69-a4f2-b7f43276b06b None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:47.905853 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-941c8c58-3daf-4f69-a4f2-b7f43276b06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:47.914370 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1408/2844] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:47.918472 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a313b1-f137-4896-a38a-11cd3963ad5b None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:47.928775 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a313b1-f137-4896-a38a-11cd3963ad5b None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:47.933384 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a156157-91b2-4a9c-8bd3-ef13f5cf2e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:47.934038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a156157-91b2-4a9c-8bd3-ef13f5cf2e34 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:47.934323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a156157-91b2-4a9c-8bd3-ef13f5cf2e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:47.934601 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a156157-91b2-4a9c-8bd3-ef13f5cf2e34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:47.935329 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a156157-91b2-4a9c-8bd3-ef13f5cf2e34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:56:47.935639 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a156157-91b2-4a9c-8bd3-ef13f5cf2e34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:47.937273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a156157-91b2-4a9c-8bd3-ef13f5cf2e34 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:47.948635 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1437/2845] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:56:47.952772 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c8c58-3daf-4f69-a4f2-b7f43276b06b None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:47.962075 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c8c58-3daf-4f69-a4f2-b7f43276b06b None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:47.967748 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb6150-8a48-4b49-be5f-6f48be70c9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:47.967748 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6150-8a48-4b49-be5f-6f48be70c9dd admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:47.967748 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6150-8a48-4b49-be5f-6f48be70c9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:47.968040 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6150-8a48-4b49-be5f-6f48be70c9dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:47.968684 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb6150-8a48-4b49-be5f-6f48be70c9dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:47.970591 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb6150-8a48-4b49-be5f-6f48be70c9dd admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:47.975484 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:48.048665 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1409/2846] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:56:47 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:56:48.053726 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a156157-91b2-4a9c-8bd3-ef13f5cf2e34 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:48.063914 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a156157-91b2-4a9c-8bd3-ef13f5cf2e34 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:48.102845 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:56:48.121326 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f75406-495b-4c00-885a-968235571c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:48.121994 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f75406-495b-4c00-885a-968235571c3c admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:48.122277 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f75406-495b-4c00-885a-968235571c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:48.122554 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f75406-495b-4c00-885a-968235571c3c admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:48.123233 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f75406-495b-4c00-885a-968235571c3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:48.125188 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f75406-495b-4c00-885a-968235571c3c admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:48.213766 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1438/2847] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:56:48 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:56:48.218070 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb6150-8a48-4b49-be5f-6f48be70c9dd admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:48.230615 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb6150-8a48-4b49-be5f-6f48be70c9dd admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:48.235246 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2bb2a-092a-41de-958f-9a8664e29d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:48.235906 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2bb2a-092a-41de-958f-9a8664e29d2f admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:48.236190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2bb2a-092a-41de-958f-9a8664e29d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:48.236463 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2bb2a-092a-41de-958f-9a8664e29d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:48.237376 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2bb2a-092a-41de-958f-9a8664e29d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:48.239468 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2bb2a-092a-41de-958f-9a8664e29d2f admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:48.249899 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1410/2848] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:56:48.254229 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f75406-495b-4c00-885a-968235571c3c admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:48.265076 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f75406-495b-4c00-885a-968235571c3c admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:48.270212 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f005f4-c8a8-46e2-8479-a54281e10897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:48.270916 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f005f4-c8a8-46e2-8479-a54281e10897 admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:48.271201 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f005f4-c8a8-46e2-8479-a54281e10897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:48.271477 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f005f4-c8a8-46e2-8479-a54281e10897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:48.272203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f005f4-c8a8-46e2-8479-a54281e10897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:48.273825 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f005f4-c8a8-46e2-8479-a54281e10897 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:48.284969 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1439/2849] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:56:48.289846 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2bb2a-092a-41de-958f-9a8664e29d2f admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:48.300428 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2bb2a-092a-41de-958f-9a8664e29d2f admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:48.305123 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff28aec-387b-45c7-bee6-4cd3bc9ed8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:48.306022 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff28aec-387b-45c7-bee6-4cd3bc9ed8de admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:48.306305 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff28aec-387b-45c7-bee6-4cd3bc9ed8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:48.306580 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff28aec-387b-45c7-bee6-4cd3bc9ed8de admin admin] PATH_INFO: `/v3/projects/9e9c5c553b604f5a8f5a287782c998c7/users/4d0c9e94f3854a37948f247ca8e09c2a/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:48.321189 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff28aec-387b-45c7-bee6-4cd3bc9ed8de admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e9c5c553b604f5a8f5a287782c998c7, user_id=4d0c9e94f3854a37948f247ca8e09c2a, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:48.323840 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff28aec-387b-45c7-bee6-4cd3bc9ed8de admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:48.369678 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1411/2850] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:56:48 2022] PUT /identity/v3/projects/9e9c5c553b604f5a8f5a287782c998c7/users/4d0c9e94f3854a37948f247ca8e09c2a/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:56:48.381323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd93b9-28b2-47e4-94de-40ae1945049a None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:48.381738 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd93b9-28b2-47e4-94de-40ae1945049a None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:48.382026 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd93b9-28b2-47e4-94de-40ae1945049a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:48.408725 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-e6dd93b9-28b2-47e4-94de-40ae1945049a None None] MFA Rules not processed for user `4d0c9e94f3854a37948f247ca8e09c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:48.409956 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-e6dd93b9-28b2-47e4-94de-40ae1945049a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:48.474870 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1440/2851] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:48.485492 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff28aec-387b-45c7-bee6-4cd3bc9ed8de admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:48.487174 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-dff28aec-387b-45c7-bee6-4cd3bc9ed8de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:48.543946 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff28aec-387b-45c7-bee6-4cd3bc9ed8de admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:48.547786 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a8589-a869-44bd-88ad-152d821839d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d1911bfac984e058b0b11bb6400e782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:48.548427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a8589-a869-44bd-88ad-152d821839d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:48.548749 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a8589-a869-44bd-88ad-152d821839d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:48.549053 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a8589-a869-44bd-88ad-152d821839d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:48.557436 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a8589-a869-44bd-88ad-152d821839d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:48.558908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a8589-a869-44bd-88ad-152d821839d6 service cinder] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:48.568207 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1412/2852] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:56:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:56:50.222803 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8511a1-095e-45c9-b31a-4651e0610434 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:50.223420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8511a1-095e-45c9-b31a-4651e0610434 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:50.223420 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8511a1-095e-45c9-b31a-4651e0610434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:50.240570 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-ca8511a1-095e-45c9-b31a-4651e0610434 None None] MFA Rules not processed for user `4d0c9e94f3854a37948f247ca8e09c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:56:50.241776 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ca8511a1-095e-45c9-b31a-4651e0610434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:50.248485 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1441/2853] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:56:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:56:51.825277 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a8589-a869-44bd-88ad-152d821839d6 service cinder] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:51.828460 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-573a8589-a869-44bd-88ad-152d821839d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:51.899313 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a8589-a869-44bd-88ad-152d821839d6 service cinder] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:51.904266 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f2053-d8f1-4ab7-9de1-928687c6e1e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ac44b3f0224e56a15993a2f29c1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:51.904945 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f2053-d8f1-4ab7-9de1-928687c6e1e6 service placement] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:51.905260 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f2053-d8f1-4ab7-9de1-928687c6e1e6 service placement] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:51.905542 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f2053-d8f1-4ab7-9de1-928687c6e1e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:51.907308 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e46f2053-d8f1-4ab7-9de1-928687c6e1e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:51.919844 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f2053-d8f1-4ab7-9de1-928687c6e1e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:51.921472 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f2053-d8f1-4ab7-9de1-928687c6e1e6 service placement] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:51.931813 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1413/2854] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:56:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:56:58.051038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8511a1-095e-45c9-b31a-4651e0610434 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:56:58.052731 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ca8511a1-095e-45c9-b31a-4651e0610434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:58.119441 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8511a1-095e-45c9-b31a-4651e0610434 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:56:58.120771 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823fa8a-d62b-49e1-830c-c5eda1f77b2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:56:58.121415 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823fa8a-d62b-49e1-830c-c5eda1f77b2b service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:56:58.121698 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823fa8a-d62b-49e1-830c-c5eda1f77b2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:56:58.121972 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823fa8a-d62b-49e1-830c-c5eda1f77b2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:56:58.123743 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-c823fa8a-d62b-49e1-830c-c5eda1f77b2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:56:58.133961 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823fa8a-d62b-49e1-830c-c5eda1f77b2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:56:58.135371 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823fa8a-d62b-49e1-830c-c5eda1f77b2b service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:56:58.143549 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1442/2855] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:56:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:57:01.927188 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=78,keystone:INSERT=2 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:57:04.586676 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f2053-d8f1-4ab7-9de1-928687c6e1e6 service placement] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:04.588174 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-e46f2053-d8f1-4ab7-9de1-928687c6e1e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:57:04.661967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f2053-d8f1-4ab7-9de1-928687c6e1e6 service placement] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:04.667565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44325a18-7a3f-4db8-989a-32ef7e10e04a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:04.668765 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44325a18-7a3f-4db8-989a-32ef7e10e04a service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:04.668765 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44325a18-7a3f-4db8-989a-32ef7e10e04a service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:04.668963 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44325a18-7a3f-4db8-989a-32ef7e10e04a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:04.670584 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-44325a18-7a3f-4db8-989a-32ef7e10e04a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:57:04.684215 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44325a18-7a3f-4db8-989a-32ef7e10e04a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:04.685857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44325a18-7a3f-4db8-989a-32ef7e10e04a service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:04.697764 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1414/2856] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:57:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:57:08.139706 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:57:10.811028 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b18472-5001-4421-9435-8bec0844b5f6 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:10.816218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b18472-5001-4421-9435-8bec0844b5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:10.816218 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b18472-5001-4421-9435-8bec0844b5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:10.829403 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-f3b18472-5001-4421-9435-8bec0844b5f6 None None] MFA Rules not processed for user `0a7c204a298c42f0bbc32fa9a6ce1f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:57:10.830534 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-f3b18472-5001-4421-9435-8bec0844b5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:57:10.886811 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1443/2857] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 22:57:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:57:11.027935 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44325a18-7a3f-4db8-989a-32ef7e10e04a service nova] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:11.029392 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-44325a18-7a3f-4db8-989a-32ef7e10e04a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:57:11.097870 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44325a18-7a3f-4db8-989a-32ef7e10e04a service nova] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:11.102765 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84003db3-b7aa-4f20-a38a-e8734aa4d0af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b321586cdb3d4d25904472aaa4adba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:11.103404 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84003db3-b7aa-4f20-a38a-e8734aa4d0af service swift] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:11.103689 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84003db3-b7aa-4f20-a38a-e8734aa4d0af service swift] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:11.103963 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84003db3-b7aa-4f20-a38a-e8734aa4d0af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:11.112074 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84003db3-b7aa-4f20-a38a-e8734aa4d0af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:11.113678 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84003db3-b7aa-4f20-a38a-e8734aa4d0af service swift] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:11.121792 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1415/2858] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 22:57:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:57:11.929518 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89304a76-c6e3-47b3-90fd-03b1393b9bf5 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:11.930012 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89304a76-c6e3-47b3-90fd-03b1393b9bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:11.930123 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89304a76-c6e3-47b3-90fd-03b1393b9bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:11.947911 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-89304a76-c6e3-47b3-90fd-03b1393b9bf5 None None] MFA Rules not processed for user `0a7c204a298c42f0bbc32fa9a6ce1f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:57:11.949145 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-89304a76-c6e3-47b3-90fd-03b1393b9bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:57:11.957090 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1444/2859] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 23 22:57:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:57:11.968857 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84003db3-b7aa-4f20-a38a-e8734aa4d0af service swift] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:11.979746 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84003db3-b7aa-4f20-a38a-e8734aa4d0af service swift] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:11.984753 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ec35b-de11-4a87-b426-268507db82af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b321586cdb3d4d25904472aaa4adba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:11.985643 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ec35b-de11-4a87-b426-268507db82af service swift] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:11.985956 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ec35b-de11-4a87-b426-268507db82af service swift] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:11.986233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ec35b-de11-4a87-b426-268507db82af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:11.994368 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ec35b-de11-4a87-b426-268507db82af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:11.996028 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ec35b-de11-4a87-b426-268507db82af service swift] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:12.004724 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1416/2860] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 22:57:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:57:12.539009 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea4cc1-c667-443e-81fe-59e2e1865f32 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:12.539355 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea4cc1-c667-443e-81fe-59e2e1865f32 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:12.539584 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea4cc1-c667-443e-81fe-59e2e1865f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:12.557997 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-2fea4cc1-c667-443e-81fe-59e2e1865f32 None None] MFA Rules not processed for user `0a7c204a298c42f0bbc32fa9a6ce1f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:57:12.559203 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-2fea4cc1-c667-443e-81fe-59e2e1865f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:57:12.566532 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1445/2861] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 23 22:57:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:57:12.578728 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ec35b-de11-4a87-b426-268507db82af service swift] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:12.589184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ec35b-de11-4a87-b426-268507db82af service swift] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:12.594272 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19212f8c-ea21-438d-b7d7-1445fe8991f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b321586cdb3d4d25904472aaa4adba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:12.595167 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19212f8c-ea21-438d-b7d7-1445fe8991f9 service swift] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:12.595452 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19212f8c-ea21-438d-b7d7-1445fe8991f9 service swift] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:12.595791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19212f8c-ea21-438d-b7d7-1445fe8991f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:12.605364 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19212f8c-ea21-438d-b7d7-1445fe8991f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:12.606992 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19212f8c-ea21-438d-b7d7-1445fe8991f9 service swift] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:12.616432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1417/2862] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 22:57:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:57:13.175377 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b0648-123c-452e-9793-cfa808e290a8 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:13.175800 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b0648-123c-452e-9793-cfa808e290a8 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:13.175943 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b0648-123c-452e-9793-cfa808e290a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:13.193714 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-6a1b0648-123c-452e-9793-cfa808e290a8 None None] MFA Rules not processed for user `0a7c204a298c42f0bbc32fa9a6ce1f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:57:13.194933 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6a1b0648-123c-452e-9793-cfa808e290a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:57:13.202104 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1446/2863] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 23 22:57:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:57:13.215278 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19212f8c-ea21-438d-b7d7-1445fe8991f9 service swift] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:13.226740 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19212f8c-ea21-438d-b7d7-1445fe8991f9 service swift] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:13.231873 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73383db4-a74d-499d-a743-f980b9352a4c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b321586cdb3d4d25904472aaa4adba71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:13.232509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73383db4-a74d-499d-a743-f980b9352a4c service swift] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:13.232848 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73383db4-a74d-499d-a743-f980b9352a4c service swift] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:13.233129 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73383db4-a74d-499d-a743-f980b9352a4c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:13.241341 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73383db4-a74d-499d-a743-f980b9352a4c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:13.242922 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73383db4-a74d-499d-a743-f980b9352a4c service swift] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:13.250878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1418/2864] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 23 22:57:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:57:15.806528 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b0648-123c-452e-9793-cfa808e290a8 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:15.868662 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b0648-123c-452e-9793-cfa808e290a8 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:15.873212 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968ffa1-489d-4096-ad25-6ac90c081ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:15.874134 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968ffa1-489d-4096-ad25-6ac90c081ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:15.874419 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968ffa1-489d-4096-ad25-6ac90c081ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:15.874713 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968ffa1-489d-4096-ad25-6ac90c081ed7 admin admin] PATH_INFO: `/v3/users/4d0c9e94f3854a37948f247ca8e09c2a` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:15.875887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968ffa1-489d-4096-ad25-6ac90c081ed7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d0c9e94f3854a37948f247ca8e09c2a)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:15.877973 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968ffa1-489d-4096-ad25-6ac90c081ed7 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:15.992792 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d968ffa1-489d-4096-ad25-6ac90c081ed7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d0c9e94f3854a37948f247ca8e09c2a'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:57:16.004744 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d968ffa1-489d-4096-ad25-6ac90c081ed7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d0c9e94f3854a37948f247ca8e09c2a'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:57:16.005416 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d968ffa1-489d-4096-ad25-6ac90c081ed7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d0c9e94f3854a37948f247ca8e09c2a'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:57:16.006021 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d968ffa1-489d-4096-ad25-6ac90c081ed7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d0c9e94f3854a37948f247ca8e09c2a'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:57:16.012964 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-d968ffa1-489d-4096-ad25-6ac90c081ed7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d0c9e94f3854a37948f247ca8e09c2a'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:57:16.059038 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1447/2865] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 22:57:15 2022] DELETE /identity/v3/users/4d0c9e94f3854a37948f247ca8e09c2a => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:57:16.063509 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84627642-7850-4c47-a854-b35ce40f37af None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:16.063986 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84627642-7850-4c47-a854-b35ce40f37af None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:16.064085 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84627642-7850-4c47-a854-b35ce40f37af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:16.082267 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-84627642-7850-4c47-a854-b35ce40f37af None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:57:16.083470 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-84627642-7850-4c47-a854-b35ce40f37af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:57:16.114094 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=100 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:57:16.141432 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1419/2866] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:57:16.151062 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968ffa1-489d-4096-ad25-6ac90c081ed7 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:16.152467 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-d968ffa1-489d-4096-ad25-6ac90c081ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:57:16.159058 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:57:16.210094 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968ffa1-489d-4096-ad25-6ac90c081ed7 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:16.213775 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7aad7-2778-431b-8a02-ddfaeac39fe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:16.214419 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7aad7-2778-431b-8a02-ddfaeac39fe5 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:16.214707 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7aad7-2778-431b-8a02-ddfaeac39fe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:16.214982 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7aad7-2778-431b-8a02-ddfaeac39fe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:16.224865 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7aad7-2778-431b-8a02-ddfaeac39fe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:16.226290 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7aad7-2778-431b-8a02-ddfaeac39fe5 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:16.235907 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1448/2867] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:57:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:57:16.494096 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84627642-7850-4c47-a854-b35ce40f37af None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:16.495584 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-84627642-7850-4c47-a854-b35ce40f37af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:57:16.508418 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84627642-7850-4c47-a854-b35ce40f37af None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:16.513218 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0b166-eb0f-4682-a4be-a738390f21ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:16.514201 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0b166-eb0f-4682-a4be-a738390f21ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:16.514565 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0b166-eb0f-4682-a4be-a738390f21ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:16.514817 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0b166-eb0f-4682-a4be-a738390f21ea admin admin] PATH_INFO: `/v3/projects/9e9c5c553b604f5a8f5a287782c998c7` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:16.516022 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0b166-eb0f-4682-a4be-a738390f21ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e9c5c553b604f5a8f5a287782c998c7)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:16.519581 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0b166-eb0f-4682-a4be-a738390f21ea admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:16.619646 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-22b0b166-eb0f-4682-a4be-a738390f21ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e9c5c553b604f5a8f5a287782c998c7', 'request_id': 'req-22b0b166-eb0f-4682-a4be-a738390f21ea'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:57:16.674707 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-22b0b166-eb0f-4682-a4be-a738390f21ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e9c5c553b604f5a8f5a287782c998c7', 'request_id': 'req-22b0b166-eb0f-4682-a4be-a738390f21ea'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:57:16.681878 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-22b0b166-eb0f-4682-a4be-a738390f21ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e9c5c553b604f5a8f5a287782c998c7', 'request_id': 'req-22b0b166-eb0f-4682-a4be-a738390f21ea'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:57:16.691963 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-22b0b166-eb0f-4682-a4be-a738390f21ea admin admin] The token cache is being invalidate because project 9e9c5c553b604f5a8f5a287782c998c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80519) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:57:16.692244 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-22b0b166-eb0f-4682-a4be-a738390f21ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:57:16.692888 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-22b0b166-eb0f-4682-a4be-a738390f21ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:57:16.694405 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1420/2868] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 22:57:16 2022] DELETE /identity/v3/projects/9e9c5c553b604f5a8f5a287782c998c7 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:57:16.699616 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a5c93-9a37-4400-8bef-c6a492334943 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:16.699918 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a5c93-9a37-4400-8bef-c6a492334943 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:16.700206 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a5c93-9a37-4400-8bef-c6a492334943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:16.717724 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-193a5c93-9a37-4400-8bef-c6a492334943 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:57:16.718950 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-193a5c93-9a37-4400-8bef-c6a492334943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:57:16.778535 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1449/2869] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:57:16.787864 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cee21-e672-4038-be39-a5986bf7217b None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:16.788287 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cee21-e672-4038-be39-a5986bf7217b None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:16.788593 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cee21-e672-4038-be39-a5986bf7217b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:16.807708 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-116cee21-e672-4038-be39-a5986bf7217b None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:57:16.809184 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-116cee21-e672-4038-be39-a5986bf7217b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:57:16.817126 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1421/2870] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:57:16.821277 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a5c93-9a37-4400-8bef-c6a492334943 None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:16.831036 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a5c93-9a37-4400-8bef-c6a492334943 None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:16.835966 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0800bb-8d84-4114-a1a3-8704a42a733f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:16.836702 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0800bb-8d84-4114-a1a3-8704a42a733f admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:16.836995 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0800bb-8d84-4114-a1a3-8704a42a733f admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:16.837278 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0800bb-8d84-4114-a1a3-8704a42a733f admin admin] PATH_INFO: `/v3/domains` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:16.838007 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0800bb-8d84-4114-a1a3-8704a42a733f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:57:16.838346 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0800bb-8d84-4114-a1a3-8704a42a733f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:16.839964 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0800bb-8d84-4114-a1a3-8704a42a733f admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:16.853352 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1450/2871] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 23 22:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:57:16.857722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cee21-e672-4038-be39-a5986bf7217b None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:16.867799 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cee21-e672-4038-be39-a5986bf7217b None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:16.872599 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab953331-5593-4bb5-b057-d1b6823c96e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:16.873327 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab953331-5593-4bb5-b057-d1b6823c96e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:16.873677 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab953331-5593-4bb5-b057-d1b6823c96e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:16.873964 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab953331-5593-4bb5-b057-d1b6823c96e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:16.874688 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab953331-5593-4bb5-b057-d1b6823c96e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:16.876614 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab953331-5593-4bb5-b057-d1b6823c96e0 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:16.964710 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1422/2872] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:57:16 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:57:16.968798 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0800bb-8d84-4114-a1a3-8704a42a733f admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:16.979363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0800bb-8d84-4114-a1a3-8704a42a733f admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:17.035965 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce3f82-dbb6-47c9-a73c-3c8efccce479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:17.036655 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce3f82-dbb6-47c9-a73c-3c8efccce479 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:17.036939 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce3f82-dbb6-47c9-a73c-3c8efccce479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:17.037217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce3f82-dbb6-47c9-a73c-3c8efccce479 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:17.038006 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce3f82-dbb6-47c9-a73c-3c8efccce479 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:17.039925 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce3f82-dbb6-47c9-a73c-3c8efccce479 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:17.127874 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1451/2873] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:57:16 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:57:17.132540 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab953331-5593-4bb5-b057-d1b6823c96e0 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:17.142810 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab953331-5593-4bb5-b057-d1b6823c96e0 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:17.147407 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e59b9f-779f-43ef-84b5-8455c24a8135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:17.148063 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e59b9f-779f-43ef-84b5-8455c24a8135 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:17.148348 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e59b9f-779f-43ef-84b5-8455c24a8135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:17.148655 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e59b9f-779f-43ef-84b5-8455c24a8135 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:17.149381 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e59b9f-779f-43ef-84b5-8455c24a8135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:17.150980 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e59b9f-779f-43ef-84b5-8455c24a8135 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:17.161206 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1423/2874] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:57:17.165112 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce3f82-dbb6-47c9-a73c-3c8efccce479 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:17.174857 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce3f82-dbb6-47c9-a73c-3c8efccce479 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:17.180088 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f7451-4e77-40e6-baa0-4a220c63e01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:17.180767 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f7451-4e77-40e6-baa0-4a220c63e01f admin admin] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:17.181050 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f7451-4e77-40e6-baa0-4a220c63e01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:17.181327 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f7451-4e77-40e6-baa0-4a220c63e01f admin admin] PATH_INFO: `/v3/roles` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:17.182043 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f7451-4e77-40e6-baa0-4a220c63e01f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:17.183610 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f7451-4e77-40e6-baa0-4a220c63e01f admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:17.194210 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1452/2875] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:57:17.198119 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e59b9f-779f-43ef-84b5-8455c24a8135 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:17.208034 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e59b9f-779f-43ef-84b5-8455c24a8135 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:17.212545 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc34cd-cb46-4363-b77e-4c0dd60beb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:17.213424 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc34cd-cb46-4363-b77e-4c0dd60beb70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:17.213711 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc34cd-cb46-4363-b77e-4c0dd60beb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:17.213994 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc34cd-cb46-4363-b77e-4c0dd60beb70 admin admin] PATH_INFO: `/v3/projects/c0a4f134a1b6433c8e3afade85914537/users/f32ae651a9ef4937a74bf87b1689043f/roles/1305747c745d49dc9a9718b7aa8aa4a9` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:17.228495 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc34cd-cb46-4363-b77e-4c0dd60beb70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0a4f134a1b6433c8e3afade85914537, user_id=f32ae651a9ef4937a74bf87b1689043f, role_id=1305747c745d49dc9a9718b7aa8aa4a9)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:17.231178 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc34cd-cb46-4363-b77e-4c0dd60beb70 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:17.299692 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1424/2876] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:57:17 2022] PUT /identity/v3/projects/c0a4f134a1b6433c8e3afade85914537/users/f32ae651a9ef4937a74bf87b1689043f/roles/1305747c745d49dc9a9718b7aa8aa4a9 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:57:17.305548 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cf896-d052-4e9e-83c1-aa0ae78171a1 None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:17.305840 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cf896-d052-4e9e-83c1-aa0ae78171a1 None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:17.306122 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cf896-d052-4e9e-83c1-aa0ae78171a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:17.324421 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-ed1cf896-d052-4e9e-83c1-aa0ae78171a1 None None] MFA Rules not processed for user `091e3a1287aa48ddab1a4bfa5166599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:57:17.325656 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-ed1cf896-d052-4e9e-83c1-aa0ae78171a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:57:17.333871 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:57:17.384047 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1453/2877] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:57:17.393341 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc34cd-cb46-4363-b77e-4c0dd60beb70 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:17.395229 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-05cc34cd-cb46-4363-b77e-4c0dd60beb70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:57:17.452510 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc34cd-cb46-4363-b77e-4c0dd60beb70 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:17.456143 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555bcf0-f1c1-42ad-8974-346a1aba1624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:17.456851 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555bcf0-f1c1-42ad-8974-346a1aba1624 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:17.457129 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555bcf0-f1c1-42ad-8974-346a1aba1624 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:17.457404 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555bcf0-f1c1-42ad-8974-346a1aba1624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:17.466360 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555bcf0-f1c1-42ad-8974-346a1aba1624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:17.467793 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555bcf0-f1c1-42ad-8974-346a1aba1624 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:17.471977 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:57:17.478480 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1425/2878] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:57:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:57:22.497162 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171270f-92ca-4de8-912f-9eee92dd086b None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:22.497665 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171270f-92ca-4de8-912f-9eee92dd086b None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:22.497796 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171270f-92ca-4de8-912f-9eee92dd086b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:22.525590 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-c171270f-92ca-4de8-912f-9eee92dd086b None None] MFA Rules not processed for user `f32ae651a9ef4937a74bf87b1689043f`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:57:22.526813 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-c171270f-92ca-4de8-912f-9eee92dd086b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:57:22.598927 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1454/2879] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:57:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:57:22.603181 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555bcf0-f1c1-42ad-8974-346a1aba1624 service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:22.615068 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555bcf0-f1c1-42ad-8974-346a1aba1624 service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:22.619987 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449c5c8-9829-4113-a031-8135fbc2f43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:22.620684 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449c5c8-9829-4113-a031-8135fbc2f43e admin admin] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:22.620975 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449c5c8-9829-4113-a031-8135fbc2f43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:22.621254 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449c5c8-9829-4113-a031-8135fbc2f43e admin admin] PATH_INFO: `/v3/projects` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:22.622135 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449c5c8-9829-4113-a031-8135fbc2f43e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:22.624190 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449c5c8-9829-4113-a031-8135fbc2f43e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:22.713443 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1426/2880] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 23 22:57:22 2022] POST /identity/v3/projects => generated 371 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:57:22.717428 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171270f-92ca-4de8-912f-9eee92dd086b None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:22.729454 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171270f-92ca-4de8-912f-9eee92dd086b None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:22.786255 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6d0b6-3a3a-4328-8735-7c03cf5a25c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:22.786933 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6d0b6-3a3a-4328-8735-7c03cf5a25c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:22.787226 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6d0b6-3a3a-4328-8735-7c03cf5a25c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:22.787509 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6d0b6-3a3a-4328-8735-7c03cf5a25c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:22.788216 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6d0b6-3a3a-4328-8735-7c03cf5a25c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:22.790184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6d0b6-3a3a-4328-8735-7c03cf5a25c7 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:22.865993 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1455/2881] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 23 22:57:22 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:57:22.876797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449c5c8-9829-4113-a031-8135fbc2f43e admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:22.883402 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449c5c8-9829-4113-a031-8135fbc2f43e admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:22.893914 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cffd8fa-3e0a-4bb0-aeda-ab670f9fd0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:22.894843 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffd8fa-3e0a-4bb0-aeda-ab670f9fd0df admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:22.895198 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffd8fa-3e0a-4bb0-aeda-ab670f9fd0df admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:22.895545 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffd8fa-3e0a-4bb0-aeda-ab670f9fd0df admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:22.896427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cffd8fa-3e0a-4bb0-aeda-ab670f9fd0df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:22.898125 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cffd8fa-3e0a-4bb0-aeda-ab670f9fd0df admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:22.914373 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1427/2882] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:57:22.915266 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6d0b6-3a3a-4328-8735-7c03cf5a25c7 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:22.926402 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6d0b6-3a3a-4328-8735-7c03cf5a25c7 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:22.931403 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12215823-96e8-4b03-b7bd-1ef8faadb602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:22.932295 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12215823-96e8-4b03-b7bd-1ef8faadb602 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:22.932612 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12215823-96e8-4b03-b7bd-1ef8faadb602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:22.932887 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12215823-96e8-4b03-b7bd-1ef8faadb602 admin admin] PATH_INFO: `/v3/projects/a89f9ea4f89a4c198904b5187632e3d7/users/2fefb62ab3a94140bee131291ba6a88a/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:22.948121 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12215823-96e8-4b03-b7bd-1ef8faadb602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a89f9ea4f89a4c198904b5187632e3d7, user_id=2fefb62ab3a94140bee131291ba6a88a, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:22.950801 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12215823-96e8-4b03-b7bd-1ef8faadb602 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:23.014387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1456/2883] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:57:22 2022] PUT /identity/v3/projects/a89f9ea4f89a4c198904b5187632e3d7/users/2fefb62ab3a94140bee131291ba6a88a/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:57:23.018369 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cffd8fa-3e0a-4bb0-aeda-ab670f9fd0df admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:23.029287 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cffd8fa-3e0a-4bb0-aeda-ab670f9fd0df admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:23.095715 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc83876-2697-4261-affa-4167d7c7d366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:23.096459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc83876-2697-4261-affa-4167d7c7d366 admin admin] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:23.097578 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc83876-2697-4261-affa-4167d7c7d366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:23.097805 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc83876-2697-4261-affa-4167d7c7d366 admin admin] PATH_INFO: `/v3/roles` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:23.100021 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc83876-2697-4261-affa-4167d7c7d366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:23.100469 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc83876-2697-4261-affa-4167d7c7d366 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:23.111107 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1428/2884] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 23 22:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:57:23.115407 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12215823-96e8-4b03-b7bd-1ef8faadb602 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:23.126635 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12215823-96e8-4b03-b7bd-1ef8faadb602 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:23.131695 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d99b7-3f15-41f1-b273-013a530cce79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:23.132671 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d99b7-3f15-41f1-b273-013a530cce79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:23.132963 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d99b7-3f15-41f1-b273-013a530cce79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:23.133245 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d99b7-3f15-41f1-b273-013a530cce79 admin admin] PATH_INFO: `/v3/projects/a89f9ea4f89a4c198904b5187632e3d7/users/2fefb62ab3a94140bee131291ba6a88a/roles/6ff7708b59984b9287cc81d46a57b4b4` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:23.135272 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d99b7-3f15-41f1-b273-013a530cce79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a89f9ea4f89a4c198904b5187632e3d7, user_id=2fefb62ab3a94140bee131291ba6a88a, role_id=6ff7708b59984b9287cc81d46a57b4b4)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:23.138032 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d99b7-3f15-41f1-b273-013a530cce79 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:23.141559 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:57:23.150091 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1457/2885] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 23 22:57:23 2022] PUT /identity/v3/projects/a89f9ea4f89a4c198904b5187632e3d7/users/2fefb62ab3a94140bee131291ba6a88a/roles/6ff7708b59984b9287cc81d46a57b4b4 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:57:28.501041 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01290e00-852b-475a-be40-2888b1c92135 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:28.501562 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01290e00-852b-475a-be40-2888b1c92135 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:28.501682 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01290e00-852b-475a-be40-2888b1c92135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:28.548059 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-01290e00-852b-475a-be40-2888b1c92135 None None] MFA Rules not processed for user `2fefb62ab3a94140bee131291ba6a88a`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:57:28.550846 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-01290e00-852b-475a-be40-2888b1c92135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:57:28.654979 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1429/2886] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:57:28 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:57:28.670447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d99b7-3f15-41f1-b273-013a530cce79 admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:28.672217 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-678d99b7-3f15-41f1-b273-013a530cce79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:57:28.795693 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d99b7-3f15-41f1-b273-013a530cce79 admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:28.801935 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03703f-e639-498d-941e-eeb4fbcb7d0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:28.802631 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03703f-e639-498d-941e-eeb4fbcb7d0b service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:28.802921 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03703f-e639-498d-941e-eeb4fbcb7d0b service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:28.803204 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03703f-e639-498d-941e-eeb4fbcb7d0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:28.866594 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd03703f-e639-498d-941e-eeb4fbcb7d0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:28.868348 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd03703f-e639-498d-941e-eeb4fbcb7d0b service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:28.879651 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1458/2887] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:57:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:57:32.373545 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9cccb7-1873-48ea-a7b9-5b2bb86cf8fe None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:32.374053 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9cccb7-1873-48ea-a7b9-5b2bb86cf8fe None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:32.374174 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9cccb7-1873-48ea-a7b9-5b2bb86cf8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:32.392171 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-ee9cccb7-1873-48ea-a7b9-5b2bb86cf8fe None None] MFA Rules not processed for user `f32ae651a9ef4937a74bf87b1689043f`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:57:32.393410 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-ee9cccb7-1873-48ea-a7b9-5b2bb86cf8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:57:32.400298 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1430/2888] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 23 22:57:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:57:38.874323 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=76 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:57:42.387174 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=96,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:57:48.103239 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03703f-e639-498d-941e-eeb4fbcb7d0b service nova] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:57:48.169214 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03703f-e639-498d-941e-eeb4fbcb7d0b service nova] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:57:48.172972 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34cca6-eb99-4647-bab2-f3010cab2ec2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:57:48.173622 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34cca6-eb99-4647-bab2-f3010cab2ec2 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:57:48.173915 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34cca6-eb99-4647-bab2-f3010cab2ec2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:57:48.174194 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34cca6-eb99-4647-bab2-f3010cab2ec2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:57:48.176951 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-7b34cca6-eb99-4647-bab2-f3010cab2ec2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:57:48.200801 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b34cca6-eb99-4647-bab2-f3010cab2ec2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:57:48.202235 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b34cca6-eb99-4647-bab2-f3010cab2ec2 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:57:48.210938 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1459/2889] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:57:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:57:58.206694 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=35 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:58:07.681827 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9cccb7-1873-48ea-a7b9-5b2bb86cf8fe None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:58:07.692121 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9cccb7-1873-48ea-a7b9-5b2bb86cf8fe None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:58:07.695655 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c478d-78ab-4379-93e9-12645a96c9ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:58:07.696293 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c478d-78ab-4379-93e9-12645a96c9ad service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:58:07.696596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c478d-78ab-4379-93e9-12645a96c9ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:58:07.696893 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c478d-78ab-4379-93e9-12645a96c9ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:58:07.705858 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c478d-78ab-4379-93e9-12645a96c9ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:58:07.707888 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c478d-78ab-4379-93e9-12645a96c9ad service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:58:07.717521 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1431/2890] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:58:07 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:58:09.835733 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34cca6-eb99-4647-bab2-f3010cab2ec2 service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:58:09.856209 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34cca6-eb99-4647-bab2-f3010cab2ec2 service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:58:09.862273 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3870ab-9785-4f08-8763-0b93aab6ec54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:58:09.862931 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3870ab-9785-4f08-8763-0b93aab6ec54 service nova] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:58:09.864262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3870ab-9785-4f08-8763-0b93aab6ec54 service nova] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:58:09.864262 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3870ab-9785-4f08-8763-0b93aab6ec54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:58:09.866002 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6d3870ab-9785-4f08-8763-0b93aab6ec54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:58:09.878244 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3870ab-9785-4f08-8763-0b93aab6ec54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:58:09.880029 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3870ab-9785-4f08-8763-0b93aab6ec54 service nova] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:58:09.890725 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1460/2891] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:58:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:58:17.713456 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=6 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:58:19.886175 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=9 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:58:26.884179 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c478d-78ab-4379-93e9-12645a96c9ad service neutron] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:58:26.885597 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-b23c478d-78ab-4379-93e9-12645a96c9ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:58:26.975799 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c478d-78ab-4379-93e9-12645a96c9ad service neutron] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:58:26.980802 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe268a-94bc-4fff-92b6-ef2a8e70c2af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ac44b3f0224e56a15993a2f29c1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:58:26.981442 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe268a-94bc-4fff-92b6-ef2a8e70c2af service placement] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:58:26.981754 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe268a-94bc-4fff-92b6-ef2a8e70c2af service placement] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:58:26.982032 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe268a-94bc-4fff-92b6-ef2a8e70c2af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:58:26.983795 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-6ffe268a-94bc-4fff-92b6-ef2a8e70c2af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:58:26.995705 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe268a-94bc-4fff-92b6-ef2a8e70c2af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:58:26.997331 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe268a-94bc-4fff-92b6-ef2a8e70c2af service placement] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:58:27.007709 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1432/2892] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 22:58:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:58:37.002876 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=41 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 23:00:08.954325 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3870ab-9785-4f08-8763-0b93aab6ec54 service nova] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 23:00:08.955851 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-6d3870ab-9785-4f08-8763-0b93aab6ec54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 23:00:08.967363 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3870ab-9785-4f08-8763-0b93aab6ec54 service nova] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 23:00:08.970844 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c4e7e-42f9-431c-b3a3-a020761ebe66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 23:00:08.971474 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c4e7e-42f9-431c-b3a3-a020761ebe66 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 23:00:08.971758 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c4e7e-42f9-431c-b3a3-a020761ebe66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 23:00:08.972033 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c4e7e-42f9-431c-b3a3-a020761ebe66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 23:00:08.973732 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-217c4e7e-42f9-431c-b3a3-a020761ebe66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 23:00:09.000008 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c4e7e-42f9-431c-b3a3-a020761ebe66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 23:00:09.001449 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c4e7e-42f9-431c-b3a3-a020761ebe66 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 23:00:09.021030 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1461/2893] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 23:00:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 23:00:19.018452 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=15 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 23:00:19.844391 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bff372-3702-40f5-b532-f85b7952634d None None] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 23:00:19.846791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bff372-3702-40f5-b532-f85b7952634d None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 23:00:19.846791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bff372-3702-40f5-b532-f85b7952634d None None] PATH_INFO: `/` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 23:00:19.846791 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1433/2894] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 23:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 23:00:19.852303 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9dec3f-fbb6-461e-942d-485e08451bdb None None] REQUEST_METHOD: `POST` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 23:00:19.852678 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9dec3f-fbb6-461e-942d-485e08451bdb None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 23:00:19.852959 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9dec3f-fbb6-461e-942d-485e08451bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 23:00:19.913833 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.auth.core [None req-8d9dec3f-fbb6-461e-942d-485e08451bdb None None] MFA Rules not processed for user `ec7412a9a3dc46369fe7b4771e3437de`. Rule list: `[]` (Enabled: `True`). {{(pid=80518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 23:00:19.915049 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-8d9dec3f-fbb6-461e-942d-485e08451bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 23:00:19.921561 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1462/2895] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 23:00:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 23:00:19.933779 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bff372-3702-40f5-b532-f85b7952634d None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 23:00:19.945282 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bff372-3702-40f5-b532-f85b7952634d None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 23:00:19.948797 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4462502-63d7-4f1b-974e-90bb88e39d42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 23:00:19.949427 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4462502-63d7-4f1b-974e-90bb88e39d42 service neutron] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 23:00:19.949712 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4462502-63d7-4f1b-974e-90bb88e39d42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 23:00:19.949988 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4462502-63d7-4f1b-974e-90bb88e39d42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 23:00:19.956978 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4462502-63d7-4f1b-974e-90bb88e39d42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 23:00:19.958516 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4462502-63d7-4f1b-974e-90bb88e39d42 service neutron] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 23:00:19.966515 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1434/2896] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 23:00:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 23:00:20.046447 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754073ae-fd6c-4da1-8a8a-fcc36790260e None None] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 23:00:20.046914 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754073ae-fd6c-4da1-8a8a-fcc36790260e None None] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 23:00:20.047053 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754073ae-fd6c-4da1-8a8a-fcc36790260e None None] PATH_INFO: `/` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 23:00:20.048406 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1463/2897] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 23 23:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 23:00:20.054271 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71cff9-d079-474e-8ade-7178fdb22447 None None] REQUEST_METHOD: `POST` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 23:00:20.054561 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71cff9-d079-474e-8ade-7178fdb22447 None None] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 23:00:20.054842 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71cff9-d079-474e-8ade-7178fdb22447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 23:00:20.090776 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.auth.core [None req-7f71cff9-d079-474e-8ade-7178fdb22447 None None] MFA Rules not processed for user `159cc8befc3a4d57bbd313832084db62`. Rule list: `[]` (Enabled: `True`). {{(pid=80519) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 23:00:20.091972 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-7f71cff9-d079-474e-8ade-7178fdb22447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 23:00:20.101533 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1435/2898] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 23 23:00:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 23:00:20.111818 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754073ae-fd6c-4da1-8a8a-fcc36790260e None None] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 23:00:20.123741 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754073ae-fd6c-4da1-8a8a-fcc36790260e None None] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 23:00:20.129294 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cad51c-9c4b-4571-97e6-384f98bb39bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 23:00:20.129294 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cad51c-9c4b-4571-97e6-384f98bb39bb service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 23:00:20.129548 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cad51c-9c4b-4571-97e6-384f98bb39bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 23:00:20.129683 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cad51c-9c4b-4571-97e6-384f98bb39bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 23:00:20.140517 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cad51c-9c4b-4571-97e6-384f98bb39bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 23:00:20.141974 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cad51c-9c4b-4571-97e6-384f98bb39bb service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 23:00:20.153256 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1464/2899] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 23:00:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 23:00:30.088307 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=15 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 23:00:30.148407 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=23 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 23:01:48.221773 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71cff9-d079-474e-8ade-7178fdb22447 None None] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 23:01:48.221773 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-7f71cff9-d079-474e-8ade-7178fdb22447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 23:01:48.236801 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71cff9-d079-474e-8ade-7178fdb22447 None None] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 23:01:48.243379 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0290a7-b104-40a3-8c0c-8d0c2fe5d9c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '159cc8befc3a4d57bbd313832084db62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 23:01:48.244039 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0290a7-b104-40a3-8c0c-8d0c2fe5d9c9 service nova] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 23:01:48.244342 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0290a7-b104-40a3-8c0c-8d0c2fe5d9c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 23:01:48.244638 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0290a7-b104-40a3-8c0c-8d0c2fe5d9c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 23:01:48.247656 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-dc0290a7-b104-40a3-8c0c-8d0c2fe5d9c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 23:01:48.261343 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0290a7-b104-40a3-8c0c-8d0c2fe5d9c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 23:01:48.262862 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0290a7-b104-40a3-8c0c-8d0c2fe5d9c9 service nova] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 23:01:48.281472 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1436/2900] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 23:01:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 23:01:48.578228 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cad51c-9c4b-4571-97e6-384f98bb39bb service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 23:01:48.588466 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cad51c-9c4b-4571-97e6-384f98bb39bb service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 23:01:48.592249 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e3a49-f30a-458e-a006-936c7cf30197 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec7412a9a3dc46369fe7b4771e3437de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 23:01:48.592943 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e3a49-f30a-458e-a006-936c7cf30197 service neutron] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 23:01:48.593232 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e3a49-f30a-458e-a006-936c7cf30197 service neutron] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 23:01:48.593515 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e3a49-f30a-458e-a006-936c7cf30197 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 23:01:48.595368 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-465e3a49-f30a-458e-a006-936c7cf30197 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 23:01:48.609999 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e3a49-f30a-458e-a006-936c7cf30197 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 23:01:48.611505 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e3a49-f30a-458e-a006-936c7cf30197 service neutron] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 23:01:48.623470 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1465/2901] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 23:01:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 23:01:56.707825 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0290a7-b104-40a3-8c0c-8d0c2fe5d9c9 service nova] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 23:01:56.720329 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0290a7-b104-40a3-8c0c-8d0c2fe5d9c9 service nova] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 23:01:56.725596 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfdaa0-2ab4-44a0-badf-9c9198dc88bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ac44b3f0224e56a15993a2f29c1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 23:01:56.726233 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfdaa0-2ab4-44a0-badf-9c9198dc88bc service placement] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 23:01:56.726557 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfdaa0-2ab4-44a0-badf-9c9198dc88bc service placement] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 23:01:56.726840 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfdaa0-2ab4-44a0-badf-9c9198dc88bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 23:01:56.728752 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-85cfdaa0-2ab4-44a0-badf-9c9198dc88bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 23:01:56.741317 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfdaa0-2ab4-44a0-badf-9c9198dc88bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 23:01:56.743067 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfdaa0-2ab4-44a0-badf-9c9198dc88bc service placement] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 23:01:56.753391 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1437/2902] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 23:01:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 23:01:58.618835 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=6 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 23:02:00.199842 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e3a49-f30a-458e-a006-936c7cf30197 service neutron] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 23:02:00.212782 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e3a49-f30a-458e-a006-936c7cf30197 service neutron] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 23:02:00.218059 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1169e-1df3-4bc2-b534-ae556526a05f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ac44b3f0224e56a15993a2f29c1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 23:02:00.218890 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1169e-1df3-4bc2-b534-ae556526a05f service placement] REQUEST_METHOD: `GET` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 23:02:00.219184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1169e-1df3-4bc2-b534-ae556526a05f service placement] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 23:02:00.219468 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1169e-1df3-4bc2-b534-ae556526a05f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 23:02:00.221319 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-75a1169e-1df3-4bc2-b534-ae556526a05f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 23:02:00.234259 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a1169e-1df3-4bc2-b534-ae556526a05f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 23:02:00.236048 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a1169e-1df3-4bc2-b534-ae556526a05f service placement] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 23:02:00.247747 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1466/2903] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 23:02:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 23:02:00.311685 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfdaa0-2ab4-44a0-badf-9c9198dc88bc service placement] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 23:02:00.393678 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfdaa0-2ab4-44a0-badf-9c9198dc88bc service placement] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 23:02:00.398140 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280aa6e-0abe-4713-81b8-068ec78db16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 23:02:00.399874 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280aa6e-0abe-4713-81b8-068ec78db16e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 23:02:00.400164 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280aa6e-0abe-4713-81b8-068ec78db16e admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 23:02:00.400442 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280aa6e-0abe-4713-81b8-068ec78db16e admin admin] PATH_INFO: `/v3/users/f32ae651a9ef4937a74bf87b1689043f` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 23:02:00.401661 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280aa6e-0abe-4713-81b8-068ec78db16e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f32ae651a9ef4937a74bf87b1689043f)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 23:02:00.403586 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280aa6e-0abe-4713-81b8-068ec78db16e admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 23:02:00.530947 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7280aa6e-0abe-4713-81b8-068ec78db16e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f32ae651a9ef4937a74bf87b1689043f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 23:02:00.543013 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7280aa6e-0abe-4713-81b8-068ec78db16e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f32ae651a9ef4937a74bf87b1689043f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 23:02:00.579513 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7280aa6e-0abe-4713-81b8-068ec78db16e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f32ae651a9ef4937a74bf87b1689043f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 23:02:00.580151 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7280aa6e-0abe-4713-81b8-068ec78db16e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f32ae651a9ef4937a74bf87b1689043f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 23:02:00.580785 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-7280aa6e-0abe-4713-81b8-068ec78db16e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f32ae651a9ef4937a74bf87b1689043f'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 23:02:00.593852 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1438/2904] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 23:02:00 2022] DELETE /identity/v3/users/f32ae651a9ef4937a74bf87b1689043f => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 23:02:00.843870 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1169e-1df3-4bc2-b534-ae556526a05f service placement] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 23:02:00.858387 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1169e-1df3-4bc2-b534-ae556526a05f service placement] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 23:02:00.912890 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13edbb-ab59-4768-9d54-b0e65610ac6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 23:02:00.913751 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13edbb-ab59-4768-9d54-b0e65610ac6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 23:02:00.914035 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13edbb-ab59-4768-9d54-b0e65610ac6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 23:02:00.914335 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13edbb-ab59-4768-9d54-b0e65610ac6d admin admin] PATH_INFO: `/v3/projects/c0a4f134a1b6433c8e3afade85914537` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 23:02:00.915539 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13edbb-ab59-4768-9d54-b0e65610ac6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0a4f134a1b6433c8e3afade85914537)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 23:02:00.917457 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13edbb-ab59-4768-9d54-b0e65610ac6d admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 23:02:01.011271 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cf13edbb-ab59-4768-9d54-b0e65610ac6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0a4f134a1b6433c8e3afade85914537', 'request_id': 'req-cf13edbb-ab59-4768-9d54-b0e65610ac6d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 23:02:01.018039 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cf13edbb-ab59-4768-9d54-b0e65610ac6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0a4f134a1b6433c8e3afade85914537', 'request_id': 'req-cf13edbb-ab59-4768-9d54-b0e65610ac6d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 23:02:01.023734 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cf13edbb-ab59-4768-9d54-b0e65610ac6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0a4f134a1b6433c8e3afade85914537', 'request_id': 'req-cf13edbb-ab59-4768-9d54-b0e65610ac6d'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 23:02:01.107934 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cf13edbb-ab59-4768-9d54-b0e65610ac6d admin admin] The token cache is being invalidate because project c0a4f134a1b6433c8e3afade85914537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 23:02:01.108222 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cf13edbb-ab59-4768-9d54-b0e65610ac6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 23:02:01.108874 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-cf13edbb-ab59-4768-9d54-b0e65610ac6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 23:02:01.110463 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1467/2905] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 23:02:00 2022] DELETE /identity/v3/projects/c0a4f134a1b6433c8e3afade85914537 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 23:02:01.114458 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280aa6e-0abe-4713-81b8-068ec78db16e admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 23:02:01.115967 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-7280aa6e-0abe-4713-81b8-068ec78db16e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 23:02:01.131564 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280aa6e-0abe-4713-81b8-068ec78db16e admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 23:02:01.165382 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 23:02:01.189395 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9e4ed-3be5-4105-9b8a-ae6f7145acc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 23:02:01.190267 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9e4ed-3be5-4105-9b8a-ae6f7145acc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 23:02:01.190554 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9e4ed-3be5-4105-9b8a-ae6f7145acc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 23:02:01.190830 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9e4ed-3be5-4105-9b8a-ae6f7145acc3 admin admin] PATH_INFO: `/v3/users/2fefb62ab3a94140bee131291ba6a88a` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 23:02:01.192017 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9e4ed-3be5-4105-9b8a-ae6f7145acc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fefb62ab3a94140bee131291ba6a88a)` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 23:02:01.193984 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9e4ed-3be5-4105-9b8a-ae6f7145acc3 admin admin] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 23:02:01.364457 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-22e9e4ed-3be5-4105-9b8a-ae6f7145acc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fefb62ab3a94140bee131291ba6a88a'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 23:02:01.376877 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-22e9e4ed-3be5-4105-9b8a-ae6f7145acc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fefb62ab3a94140bee131291ba6a88a'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 23:02:01.424836 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-22e9e4ed-3be5-4105-9b8a-ae6f7145acc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fefb62ab3a94140bee131291ba6a88a'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 23:02:01.425459 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-22e9e4ed-3be5-4105-9b8a-ae6f7145acc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fefb62ab3a94140bee131291ba6a88a'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 23:02:01.426263 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.notifications [None req-22e9e4ed-3be5-4105-9b8a-ae6f7145acc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fefb62ab3a94140bee131291ba6a88a'} {{(pid=80519) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 23:02:01.439062 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1439/2906] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 23 23:02:01 2022] DELETE /identity/v3/users/2fefb62ab3a94140bee131291ba6a88a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 23:02:01.655985 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13edbb-ab59-4768-9d54-b0e65610ac6d admin admin] Authenticating user token {{(pid=80518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 23:02:01.657510 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.fernet_utils [None req-cf13edbb-ab59-4768-9d54-b0e65610ac6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 23:02:01.671005 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13edbb-ab59-4768-9d54-b0e65610ac6d admin admin] Validating token access rules against request {{(pid=80518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 23:02:01.727480 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc472e-2376-443c-a2f6-c8bd6ad76d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091e3a1287aa48ddab1a4bfa5166599f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0d9b0cc6e44015a1eedda4627f582b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 23:02:01.728354 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc472e-2376-443c-a2f6-c8bd6ad76d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 23:02:01.728671 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc472e-2376-443c-a2f6-c8bd6ad76d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 23:02:01.728951 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc472e-2376-443c-a2f6-c8bd6ad76d40 admin admin] PATH_INFO: `/v3/projects/a89f9ea4f89a4c198904b5187632e3d7` {{(pid=80518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 23:02:01.730141 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc472e-2376-443c-a2f6-c8bd6ad76d40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a89f9ea4f89a4c198904b5187632e3d7)` {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 23:02:01.732034 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc472e-2376-443c-a2f6-c8bd6ad76d40 admin admin] RBAC: Authorization granted {{(pid=80518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 23:02:01.739804 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 23:02:01.808184 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-9efc472e-2376-443c-a2f6-c8bd6ad76d40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a89f9ea4f89a4c198904b5187632e3d7', 'request_id': 'req-9efc472e-2376-443c-a2f6-c8bd6ad76d40'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 23:02:01.814131 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-9efc472e-2376-443c-a2f6-c8bd6ad76d40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a89f9ea4f89a4c198904b5187632e3d7', 'request_id': 'req-9efc472e-2376-443c-a2f6-c8bd6ad76d40'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 23:02:01.819767 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-9efc472e-2376-443c-a2f6-c8bd6ad76d40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a89f9ea4f89a4c198904b5187632e3d7', 'request_id': 'req-9efc472e-2376-443c-a2f6-c8bd6ad76d40'} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 23:02:01.881344 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-9efc472e-2376-443c-a2f6-c8bd6ad76d40 admin admin] The token cache is being invalidate because project a89f9ea4f89a4c198904b5187632e3d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 23:02:01.881633 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-9efc472e-2376-443c-a2f6-c8bd6ad76d40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 23:02:01.882245 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG keystone.notifications [None req-9efc472e-2376-443c-a2f6-c8bd6ad76d40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 23:02:01.884383 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: [pid: 80518|app: 0|req: 1468/2907] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 23 23:02:01 2022] DELETE /identity/v3/projects/a89f9ea4f89a4c198904b5187632e3d7 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 23:02:11.435612 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=37,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 23:02:11.878123 dv-cinder-813229-11-iscsi devstack@keystone.service[80518]: DEBUG dbcounter [-] [80518] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 23:03:33.828599 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9e4ed-3be5-4105-9b8a-ae6f7145acc3 admin admin] Authenticating user token {{(pid=80519) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 23:03:33.830274 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-22e9e4ed-3be5-4105-9b8a-ae6f7145acc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 23:03:33.898274 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9e4ed-3be5-4105-9b8a-ae6f7145acc3 admin admin] Validating token access rules against request {{(pid=80519) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 23:03:33.956697 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fcc885-832a-4e53-8219-f1843cf10e68 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ac44b3f0224e56a15993a2f29c1e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '14265887fea245a5a37abbb3a116e352', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80519) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 23:03:33.957344 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcc885-832a-4e53-8219-f1843cf10e68 service placement] REQUEST_METHOD: `GET` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 23:03:33.957629 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcc885-832a-4e53-8219-f1843cf10e68 service placement] SCRIPT_NAME: `/identity` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 23:03:33.957908 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcc885-832a-4e53-8219-f1843cf10e68 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80519) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 23:03:33.959663 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.fernet_utils [None req-02fcc885-832a-4e53-8219-f1843cf10e68 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80519) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 23:03:33.971220 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fcc885-832a-4e53-8219-f1843cf10e68 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 23:03:33.972809 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fcc885-832a-4e53-8219-f1843cf10e68 service placement] RBAC: Authorization granted {{(pid=80519) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 23:03:33.982722 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: [pid: 80519|app: 0|req: 1440/2908] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 23 23:03:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 23:03:43.978322 dv-cinder-813229-11-iscsi devstack@keystone.service[80519]: DEBUG dbcounter [-] [80519] Writing DB stats keystone:SELECT=58 {{(pid=80519) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}